Scope of the wiki page: description of the online applications for AdV. It concerns all applications that need to process Virgo (and LIGO) data with low latency. Latency can be as low as few seconds up to several hours.

The goal of this table is to provide as much information as possible to size the needed upgrades of the Cascina computing facilities (farm clusters, data storage, data access, network connection, disk storage, internet access, web server, ...). This will also serve as primary information to discuss progresses and problems of each project in VDAS calls.

In the column "core", users should provide a reference node for their estimation. For instance, reference cores have different power:
"olnode" core (olnode33-75): 1 core from opteron 275 2.2GHz 1MB L2 cache , 1GHz bus - (year 2005) - 6 HS06
"olserver" core (olserver1-10,15-18): 1 core from Intel E5450 3.0GHz  12MB L2 cache 1.33GHz bus - (year 2007) - 26 HS06

The column "commissioning / detector operation" is meant to provide information about the relation of the applications output and the commissioning and the detector operation.

Application name

Data access (input/bandwidth)

Input data

Output data

Output volume

Receiving LIGO data

Use of special service

Processes number / duration [fraction of day]

Cores number / type

RAM

Need of batch scheduler at cascina?/ type of implementation

Reprocessing possibility / which data stream?

Commissioning / detector operation

Comments

MonitoringWeb

mainly trend.ffl about 0.5 MB/s

trenddata

web pages (web server)

100 MB/day on disk

no, except some use of wget

VDB, channelsDB, Cl web pages

200 processes 5h/day

6 cores like those of olserver

1 GB per core

no / Md and bash scripts

yes / mainly trenddata

monitoring &  "several minutes latency" alarm

Virgo summary pages

Spectrograms

DAQ shared memory (but no real need for low latency)

spectro data GWF files

web pages (web server)

20 MB/day on disk

no

Spectro on Demand (do spectrograms via web user interface)

360 processes 20h/day

8 cores like those of olserver

1 GB per core

maybe / Md and bash scripts + on demand

yes / trenddata + spectro files

commissioning

Spectrograms

DQ monitoring, hardware injection, safety,

raw.ffl about 0.2 MB/s

- VDB - some raw data channels

- web pages (web server)

300 MB/day on disk

no

VDB

?

6 cores like those of olserver

1 GB per core

no / ?

yes / raw data and VDB content

monitoring

Provide online and offline information about DQ flags

DQ performance

- Omicron triggers files stored on disks

- VDB - Omicron triggers files

- web pages (web server)

100 MB/day on disk

no

VDB

1 process 2h/day

1 olnode

4 GB /core

no / cron and on demand

yes / Omicron triggers

diagnostic

performance of DQ and vetoes

DQ production (hVeto, Bas, PQ, BRMSMon, ...)

- DAQ shared memory - 50 Hz andtrenddata files stored on disks

- Omicron triggers files - some raw data channels - VDB?

- segments to be uploaded in VDB/segdb

1 MB/day on disk

no

?

14 olserver?

1 GB/core

no /

diagnostic

veto production

UPV

- Omicron triggers files

- files on disk - VDB

- files on disk (web server) - VDB

A few MB/day on disk

no

VDB

1 process

2 olnode

4 GB  /core

no / ?

yes / Omicron triggers

DA

veto production

Omicron

- DAQ shared memory

- ~400 raw data channels

- Omciron triggers files (root format) on disk - 1 file per channel written each 12 s

5 GB/day (13 MB / channel / day)

yes

no

30-40 processes

- ~40 olnode - multi core machines is favored

4 GB / core

no / DAQ

yes / raw data

monitoring

multi-channel transient triggers production

Omega scan

- raw data files (1 file at a time)

- ~200 channels

~20 MB / scan

- few scans per day (100 MB / day)

no

1 or 2 processes 2-4h/day

1-2 olnode

4 GB / core

no / on demand only

no

investigations

multi-channel scan. OmiScan should be preferred

OmiScan

- omicron triggers files

- files on disk

- files on disk (web server)

- 5 MB / scan - few dozen scans / day --> 100 MB / day

no

channel DB

several hours/day

2-4

4 GB / core

no / user scripts and on demand

no

investigations

substitute for Omega scans: faster and lighter

SpectroMoni

- DAQ shared memory - raw data files on disk

~30 channels?

spectro GWF files on disk

1.2 GB  / day

no

no

1 process 24h/day

2 cores like those of olserver

2 GB / core

no / DAQ

yes / raw data on disk

compute spectra for spectrograms

DMS, noise monitoring

- DAQ shared memory

~ 120 channels

- DMS database - files on disk (web server) - DAQ stream

- 10 MB / day?

no

?

8 cores like those of olserver

2 GB / core

no / DAQ

yes / raw data on disk

monitoring, alarm, vetoes

all processes involved in detector monitoring and online noise monitoring

dataDisplay provider, dataSender

- DAQ shared memory - raw data files on disk

5-10 MB/s

none

NA

no

channelsDB?

2 main processes 24h/day - 5-10 sub-processes (one per data request) few hours/day

16 cores like those of olserver

2 GB / core

no / DAQ

NA

commissioning, monitoring, investigations

server for access to DAQ streams or raw data files at Cascina and remotely

DQ developements and tests

- raw data files on disk

dozen of channels

- files on disk

- 100-500 GB

no

no

few processes 24h/day few days/month

8 cores like those of olnode

2 GB / core

no / ??

NA

investigations

spare machines used for tests of DQ tools

NoEMi

raw.ffl

~100 raw data channels

Files, LinesDB entries, web pages

- Files: few GB/day - DB: few MB/year -Web pages: few MB/day

no

MySQL accessible to read and write results

~100 processes 2-3 hours/day

1 core/process

1 GB/core in general, 3 GB/core for the 20KHz sampled channels

will help

no

Monitoring during commissioning & science runs

MBTA

FdIO 100kB/s

low latency h(t) (include state vectors)

Frame files, graceDB web pages

~1GB/day

yes

Cm

~40 (ER3 configuration)

32 olserver40

128GB

no

yes

DQ studies and  science run

Online data transfer

FdIO 100kB/s

low latency h(t)

Files, web pages

10GB/day

yes

Cm

few

1

small

no

no

Offline data transfer

h reconstruction

DAQ

10-20 raw channels

DAQ, files

5GB/day

no

Cm

few

few

small

no

yes

DNM (API)

disk/ web

web,files

?

yes

MySQL accessible to read and write results

1core+distributed system for scripts

maybe

Monitoring during commissioning & science runs

Noise monitors web interface

Coherence

raw.ffl

All Channels

MySQL DB

to be defined (depends on max bandwidth)

MySQL accessible to read and write results

test

test

will help

Monitoring during commissioning & science runs

HOSA

raw.ffl

to be tested

files or DB integrated in DNM(API)

TBD

test

test

will help

Monitoring during commissioning & science runs

Non linear system identification SFOS

raw.ffl

~500 channels

web pages (?)

TBD

No

do analysis via web (user interface)

tests

tests

tests

tests

yes

Monitoring  and analysis for commissioning & science runs

under test

Bilinear coupling monitoring

raw.ffl

~100 channels

TBD: integrated in DNM(API)

TBD

No

TBD

o(10-100)

TBD

TBD

yes

yes

Monitoring during commissioning and science run

NonStatMoni

raw.ffl

TBD

integrated in DNM(API)

TBD

TBD

TBD

will help

Monitoring during commissioning and science run