Virgo daemons Management
Procedures for Virgo Daemons Restarting/Starting/Checking
Restarting
This is the case in which the Daemon is misbehaving or denying
user correct actions but is still receiving Cm messages.
Here the restart-commands to be run (note: they can all be issued on any control room workstation by any user):
CmNameServer: issue "cm restart_server"
Database Server: issue "db restart_server"
Error Logger Server: issue "el restart_server"
Virgo Main Master Server: issue "vmm restart_server"
Starting
This is the case in which the server has crashed or do not respond
anymore to any Cm message (like the restart).
The important point to be taken in account is that the daemons
(*.exe) are started by a start script (*.startcascina) that
automatically restarts them in case they crash.
Login into the machine olserver as virgod (note: for CascinaVTF or CascinaVCS you can also log with your normal user account at ctrlvtf/olvtf, respectively):
ssh olserver -l virgod
To check if the start scripts run:
ps -ef | grep start
And this is the output if all are running:
0 S virgod 19803 1 0
69 0 - 1066 wait4 Jul12
? 00:00:00 /bin/sh
/virgoApp/Cm/v7r16/mgr/NameServer.startcascina Cascina
0 S virgod 20137 1 0
69 0 - 1177 rt_sig Jul12
? 00:00:00 /bin/csh -f
/virgo/VCS-2.0/VIRGOSW//El/v4r7/mgr/ELServer.startcascina Cascina
0 S virgod 20143 1 0
69 0 - 1177 rt_sig Jul12
? 00:00:00 /bin/csh -f
/virgo/VCS-2.0/VIRGOSW//El/v4r7/mgr/ESServer.startcascina Cascina
0 S virgod 20511 1 0
69 0 - 1177 rt_sig Jul12
? 00:00:00 /bin/csh -f
/virgo/VCS-2.0/VIRGOSW//VMM/v1r6/mgr/VMMServer.startcascina Cascina
0 S virgod 6130 1
0 69 0 - 1187 rt_sig Jul28
? 00:00:00 /bin/csh -f
/virgo/VCS-2.0/VIRGOSW//Db/v4r8/mgr/DbServer.startcascina
To check if the daemons are running:
ps -ef | grep exe
And this is the output if all are running:
0 S virgod 19840 19803 0 70
0 - 876 do_sel Jul12
? 01:07:27 NameServer.exe
Cascina v7r16
0 S virgod 20141 20137 0 69
0 - 1044 do_sel Jul12
? 00:01:20 ErrorLogger.exe
Cascina Cascina
0 S virgod 20147 20143 0 69
0 - 2328 do_sel Jul12
? 00:01:08 ErrorScanner.exe
Cascina Cascina
0 S virgod 20517 20511 0 69
0 - 1718 do_sel Jul12
? 00:00:30 VMMServer.exe
Cascina
0 S virgod 6155 6130 0
69 0 - 1354 do_sel Jul28
? 00:02:02 DbServer.exe
Cascina DbServerv4
There can be then different cases:
1) Both start scripts and daemons (exe) are not running
Start the start scripts and the daemons via the following commands (note: If you wish to start either the CascinaVTF or CascinaVCS domains (both are located on different machines to the Cascina domain) use the same syntax as listed below, but replace "Cascina" with "CascinaVTF" or "CascinaVCS").
$CMROOT/mgr/NameServer.startcascina Cascina&
$DBROOT/mgr/DbServer.startcascina Cascina&
$ELROOT/mgr/ELServer.startcascina Cascina&
$ELROOT/mgr/ESServer.startcascina Cascina&
$VMMROOT/mgr/VMMServer.startcascina Cascina&
2) Deamons are running but start scripts are not
Kill the daemons (exe) manually, then proceed as indicated in point 1).
3) Daemons are not running but start scripts are
Kill the start scripts, the proceed as indicated in 1).
4) Both daemons (exe) and start scripts are running
Kill the daemons (exe) and let the start scripts restart the daemon. If this is not working kill the start script and proceed as indicated in 1).
Checking
Those are the commands to be run in order to check the correct
Restarting/Starting. They can all be issued on any Control Room
workstation.
CmNameServer:
cm names
To check if the server is up and running.
Database Server:
db load_object Virgo/Partition/OSF
To check if the database is accessible.
Error Logger Server:
Run "elui" and verify that the GUI reports on its lower right corner
the text "ELErrorLogger <version>"
Virgo Main Master Server:
Run "vmmui" and verify that the GUI reports the text "VMMServer <version>"
Remember that all daemons are monitored via BigBrother. You may want to
check the trend status under the Virgo Daemon subpage.
dampingBackup.sh
On olserver is currently also running the process bacukpping all the damping cards at any download on DSP.
Such process must be running al the time. If it's not running it must be restarted as:/virgoApp/VirgoBackup/v2r2/mgr/dampingBackup.sh > /scratch.local/local/dampingBackup.log &
This process is monitored in BB.