Discussion on the CM and IP for AdV. September 5th, 2013. ============================================================ Participants: =============== J-Y Pia Florent Elena Didier Alberto Cristiano Antonella Stefano Andrzej Chris Didier Loic Federico Gergely Pia: ask, to you all, comments and an update on the status of your work in filling the sections assigned to you or your groups. For both the documents, CM an IP. Let me remind that we have been asked to have a version 1.0 of the CM by the October STAC, while the IP can by October be still in some preliminary draft form. I would ask the approval of the VSC for the CM by October 9th, so to be able to send the document to the STAC by mid-october, which means that the VSC should have received it by the end of September. Another important point is authorlist: as in the Virgo CM I will ask to put names of authors (for the collaboration). LINKS: https://workarea.ego-gw.it/ego2/virgo/data-analysis/adv-computing-model/link-to-the-svn-packages-for-the-cm-and-for-the-ip/ (link to the SVN packages. PDF files as they appear today. And STAC and ECC May 2013 reports. These are very important to understand key points which needs to be clarified) Didier: not yet read carefully the CM. Onlye a quick look.But a few comments already: Part 1: Workflows Description for commissioning and operation is short. while for detchar and science data analysis it is much longer. Shouldn't we have much shorter detchar and science data analysis sections, keeping only what is pertinent for CM? Part 2: Data model Do we need it or would it be better to reduce it and mix it with Part III (Data Management, Distribution and Access) ? Especially by mixing with Introduction of part III which seems nice. Frame Format description maybe not needed. Detchar and commissioning mixed here while separated in Workflow description. Any reason for that? In 2.4 : many tables and details. We may need in CM only the summary table of 2.5. General comment: we should try to start with a quite short document. This is a living document that will evolve. Pia: this is what was asked. That is a structure where each part focus on a different aspect of the various things. Part I and II required many hours of teleconf. and discussions with ECC. I agree Part II is long a boring but I think that the idea of a static description of the data was indeed usefull. Trying to use the same names for the same data or pipelines everywhere in the document. But I clearly agree on some reorganization of the material. In particular what you have noticed on Commssioning/Detchar organization, which should be the same. Didier: also a comment on part 3: Data management, distribution and access. 3.2 description of data management and archiving at EGO and description of storage at Computing centers seem to decoupled. We should try to define in a single description the way we want to manage data at EGO and the way we want to transfer and manage them at CCs. Pia: Yes, this is important . Notice the STAC/ECC comment against the need to keep 6 months of data in Cascina. I agree this is what we want and we need to clarify even in the CM why we do strongly support this request. Antonella and myself have already supported this request in the discussions, also with the Council. Loic,Didier: **reasonings on why 6 months of raw data is needed at Cascina** Stefano: time distribution of the data was not clear to the Council. Clarify the needs for data like commissioning, astrowatch, ... Are we going to do differently from what done in Virgo. Important to clarify this in the document. Didier: some arguments for 6 months of raw data come from DETCHAR studies on 2-3 months old data while we are in a science run over 6 months and create new DQ flags or mitigate source of glitches during the run. Pia: agree. Let me ask to Didier and Loic to try to fix this in the document. Florent: we need to organize a good Data Acces model. Pia: agree ! Didier : Part 4: DA software. I suggest to move 5.5/5.6/5.7 sections (dealing with Data Transfers, Frame Storage and Databases) in part 3 where we have data management. Pia: yes, I agree. The information in Part 3 and the (new) part 4 can be better reorganized. Stefano: from the ECC I have understood they wanted to understand what kind of software is used, e.g. if we use any commercial software Didier: it is good to have orientations from ECC but is this CM a plan for us or for the ECC ? Pia: I have tried to keep from their suggestions and (huge) experience the ideas and suggestions which seemed good for us. Let me add that it is even possible we will decide to cut some of the information I have put in the Software tables, if we will decide these are too many details. Or, some of these info might be then moved to the IP. Florent: CM document is long and hard to digest. Need to summarize a bit more. Moreover, a given topic (like data storage) goes into different places and we need to read several sections to get an overview about the topic. Pia: I understand it's long/hard and might be simplified. But the fact the info is spread in different parts is intentional. E.g.: Part 2 gives a static vision of the data. Part 3 then describes how we distribute and access the data described in Part 2. J-Y: we have to define our document. Not to pay too much attention to ECC. General scheme should be maintained. Pia: Maybe Florent and Didier could make on the document some of the reorganization they propose. Didier: we give our comments and suggestions, but only one person should be responsible for the CM plan. Pia should take final decision on CM plan. J-Y: written suggestions ? Didier: we can send to Pia some written clearer and detailed suggestions. Florent: I can work for a proposal for Section 4 Pia: Great. I will do. With help from Gergely (who wrote in the chat he volunteers to help) Gergely: yes, I can help. Pia: Great. Thanks to everybody. Now we need to know where the chairs are with filling the required information Chris: level of details ? Pia: not so deep. E.g. specify if you need any commercial software, or if you need LaLLaLApp in place. The idea is also to have similar tables for different kind of software, this to simply the reading of the document (another suggestion for ECC). Thus is something does not apply to your pipeline you can leave it blanck or put '-'. also milestones: if everything is in place for the final goal you don't have to write anything. Chris: ok, will do my part. Andrzej: offline analysis for CW filled. Work on Section 5. Do within one week. Pia: Ask by e-mail to Giovanni, Tania, Elena (present but the mic was not working) Comments ? Questions ? J-Y do you think this organization and plans are reasonable ? J-Y: yes Pia: we will next also discuss on the Implementation Plan (IP). Let me say now that the proposed index is probalby not the best. I have already received comments from Antonella/Stefano. So, start thinking about... In any case the information I have asked for is needed. We have to decide the level of details.