Attendees

Frederique, Romain, Alberto, Cristiano, Giuseppe, Livio, Didier, MAB, Gegerly

Agenda

Pending action items

- S6 data transfer to CC

- Data migration Castor --> GEMSS at CNAF

- virgo_gpfs3 disk increase

- datafind at CNAF

Projects

- Data transfer using lcggrid (Alberto) VIR-0612A-10 (pdf)

- Pilot job project status (Gergely)

- Discussion: data access, tests, planning

Minutes (MAB)

Pending action items:

1/ data transfer to CCs:

- S6 data: all transfered to Bologna. Need to be checked and moved + Livio will transfer them to Lyon with SRB from CNAF.

action item for MAB: check files and move files at Bologna before Livio sends them to Lyon

- VSR2 RDS in Bologna: storage/gpfs_virgo4/virgo/DATA/VSR2/RDS/ (not complete).

- VSR3 RDS: scheduled in VSR3/RDS/

2/ Castor -> GEMSS at CNAF? no news

action item for MAB: wake up people (Antonella, Karen, Luca, ...)

3/ gpfs3_virgo disk increase? no news. still 15 TB

action item for MAB: wake up people (Bologna)

4/ datafind at CNAF:

- Romain: want to be able to run Virgo data VSR2+VSR3. Jobs part of a workflow (DAG) defined on LSC clusters. some "datafind" jobs are run at Bologna. Scott made it run once. datafind server needs to know where are the data --> Need to agree VSR2/3 rawdata.

action item for MAB: provide the list of VSR2/VSR3 rawdata files at Bologna

action item for Gergely: update datafind server with this info (S6 location is not yet stabilized. Wait for MAB checks)

Projects

Alberto slides.

data transfer done today: SRB (Lyon) and bbftp (CNAF) and LDR. Bookeeping done by hand!

proposal : solution based on gridFTP (lcg-cp provides checksum). No other GRID service

Livio: DAQ interface: no direct connection with DAQ, but to the local data management system build by Livio (.ffl file generation).

Livio: take files directly to DAQ could avoid data transfer to be stopped when local storage farm is stopped. For rawdata there are 2 storage streams. The main one and the backup one. This info would be kept in the "stream flag" (0 or 1) in the data transfer. The stream flag could also be used for flagging the reprocessing version

MAB: DT must be able to handle multi-stream files production: 50Hz, tredn, raw, proc, RDS ... generated online or offline. Not understood how SE definition takes into account the fact there are mutiple streams transfered in parallel.

Tests done: Between Rome and XX. Cascina and XX. in total 3 weeks.

User interface in CC:

CNAF: files should be easily accessible by users using GRID tools and Storm interface

IN2P3: to access files transfered trough xrootd, one needs to use dCache/Xrootd interface. A priori, atlas has tested this. Lyon warned the lack of manpower to provide user support

Action items: Gergely for CNAF and MAB for CCIN2P3 will build up small benchmark programs to test the access of files transfered by VDT.

Alberto: files transfered at CNAF have been deleted because they were on virgo_gpf3. Will re-transfer some.

VDB interface: no time to discuss this. Postpone to another meeting

Next meeting: Nov 17 during Virgo week. Agenda: action items follow up and pilot jobs status.

Email from Alberto after the call:

hi Gergely and Marie-Anne,

thanks for volunteering to test the data access... :) As I told, I had to remove the replicas of the raw data I've transferred to Cnaf due to lack of space on the user area (/gpfs_virgo3). The data is still present at in2p3 and in the LFC:

/grid/virgo/TSCascina/raw/0


$ lcg-lr lfn:/grid/virgo/TSCascina/raw/0/V-raw-957514320-180.gwf
srm://ccsrm02.in2p3.fr/pnfs/in2p3.fr/data/virgo/tape/TSCascina//raw/0/V-raw-957514320-180.gwf


I can replicate the files at Cnaf, or transfer a new set of raw data to Cnaf and Lyon... let me know what you prefer.


Some days ago I've also transferred some tens of 50Hz data to Cnaf and Lyon, just to check the system again:

$ lfc-ls /grid/virgo/TSCascina/50Hz/0
V-973080000-06-Nov-2010-13h00-720F.50
V-973087200-06-Nov-2010-15h00-720F.50
V-973094400-06-Nov-2010-17h00-720F.50
...


$ lcg-lr lfn:/grid/virgo/TSCascina/50Hz/0/V-973080000-06-Nov-2010-13h00-720F.50

srm://ccsrm02.in2p3.fr/pnfs/in2p3.fr/data/virgo/tape/TSCascina//50Hz/0/V-973080000-06-Nov-2010-13h00-720F.50
srm://storm-fe-virgo.cr.cnaf.infn.it/virgo3/TSCascina//50Hz/0/V-973080000-06-Nov-2010-13h00-720F.50

in Bologna the files are accessible locally:

$ ls /storage/gpfs_virgo3/virgo/TSCascina/50Hz/0/
V-973080000-06-Nov-2010-13h00-720F.50 V-973188000-07-Nov-2010-19h00-720F.50
...

as you see the SRM path of the files returned by the lcg-lr command can be easily "translated" to the path on disk:

srm://storm-fe-virgo.cr.cnaf.infn.it/virgo3/ -> /storage/gpfs_virgo3/virgo/


Should I transfer some other kind of data, more files...?

cheers,

        Alberto