Differences between revisions 21 and 22
Revision 21 as of 2016-07-29 14:57:18
Size: 15178
Editor: AdrianIrles
Comment: instructions for july/august testbeam
Revision 22 as of 2016-07-29 15:05:54
Size: 15216
Editor: AdrianIrles
Comment:
Deletions are marked like this. Additions are marked like this.
Line 285: Line 285:
Line 286: Line 287:
Line 287: Line 289:
cd /home/calice/TB2016/scripts
source manual_kill.sh

cd /home/calice/TB2016/scripts source

manual_kill.sh
Line 290: Line 295:
Line 291: Line 297:
Line 292: Line 299:
Line 293: Line 301:
Line 294: Line 303:
Line 295: Line 305:
Line 296: Line 307:
Line 297: Line 309:
Line 298: Line 311:
Line 299: Line 313:
dqm4hep_start_analysis_module -f /home/calice/TB2016/DQM4HEP-AHCAL/DQM4ILC/source/examples/AHCALRawModule/TestbeamJuly2016_15channels/lcio15channelsX4_test.xml
dqm4hep_start_analysis_module -f /home/calice/TB2016/DQM4HEP-AHCAL/DQM4ILC/source/examples/AHCALRawModule/TestbeamJuly2016_15channels/lcio15channelsX4.xml
Line 301: Line 317:
Line 302: Line 319:
Line 303: Line 321:
Line 304: Line 323:
For every new position, we only need to: stop the run, import the run configuration, change channels numbers in the run control, start the run and stream the lcio file. If we are interested in hit maps, only start stop is needed (not the loading of the conf)
For every new position, we only need to: stop the run, import the run configuration, change channels numbers in the run control, start the run and stream the lcio file.

If we are interested in hit maps, only start stop is needed (not the loading of the conf)

AHCALMonitoring within the DQM4HEP framework

Contents

DQMHEP

Installation (Ubuntu 14)

ILCSoft superbuild

Check this page to build easily ILCSoft : http://flcwiki.desy.de/ILCSoftInstall

ILCSOFT prerequisites

/!\ Check out here to simplify installation -> http://flcwiki.desy.de/ILCSoftInstall /!\

Instructions to install ILCsoft in Ubuntu 14 LTS release.

We want to install the v01-17-09 ILCsoft release, for that we want to use c++11, and therefore some packages need to be installed previously.

Boost 1.58 (installation script in the link)

Other requirements (from http://www.desy.de/~rosem/howtos/howto_install_ilcsoft_on_ubuntu.txt)

###########################
# (1) System preparation: #
###########################

In order to be able to build and use ilcsoft at all; vanilla Ubuntu needs several packages.

#--- mysql includes etc.
sudo apt-get install libmysqlclient-dev

#--- create a links for mysql (if missing):
sudo ln -s /usr/lib/x86_64-linux-gnu/ /usr/lib64
sudo ln -s /usr/lib64/libmysqlclient.so.18  /usr/lib64/libmysqlclient.so

#--- install code mgmt and build tools:
sudo apt-get install subversion
sudo apt-get install cvs

sudo apt-get install cmake
sudo apt-get install cmake-curses-gui   #ccmake

sudo apt-get install doxygen
sudo apt-get install latex2html

#--- graphics stuff (X and OpenGL):
sudo apt-get install libxext-dev
sudo apt-get install freeglut3-dev
sudo apt-get install libxmu-dev libxi-dev

#--- ROOT stuff
sudo apt-get install dpkg-dev
sudo apt-get install libxpm-dev
sudo apt-get install libxft-dev
sudo apt-get install python2.7-dev

#--- javac:
sudo apt-get install  default-jdk

#--- if needed/wanted: cernlib and gfortran
sudo apt-get install cernlib
sudo apt-get install gfortran

# ---- liblog4cxx10
sudo apt-get install liblog4cxx10-dev

OR: in single line:
sudo apt-get install libmysqlclient-dev subversion cvs cmake cmake-curses-gui doxygen latex2html libxext-dev freeglut3-dev libxmu-dev libxi-dev dpkg-dev libxpm-dev libxft-dev python2.7-dev default-jdk cernlib gfortran liblog4cxx10-dev && sudo ln -s /usr/lib/x86_64-linux-gnu/ /usr/lib64 && sudo ln -s /usr/lib64/libmysqlclient.so.18  /usr/lib64/libmysqlclient.so

It might be also a good idea to install all root requirements:

sudo apt-get install git dpkg-dev make g++ gcc binutils libx11-dev libxpm-dev \
libxft-dev libxext-dev


sudo apt-get install gfortran libssl-dev libpcre3-dev \
xlibmesa-glu-dev libglew1.5-dev libftgl-dev \
libmysqlclient-dev libfftw3-dev cfitsio-dev \
graphviz-dev libavahi-compat-libdnssd-dev \
libldap2-dev python-dev libxml2-dev libkrb5-dev \
libgsl0-dev libqt4-dev

Installation of ILCSOFT

The first is to download the code (from desy).

svn co https://svnsrv.desy.de/desy/ilctools/ilcinstall/tags/v01-17-09 ilcinstall-v01-17-09

To install it, you will need to make few changes in the release-ilcsoft.cfg and release-versions.py files in

lcinstall-v01-17-09/releases/v01-17-09/

Also the ilcsoft/qt.py and ilcsoft/root.py need some changes:

  • to find the proper link to download the versions (qt and root)
  • to install root with all needed dependencies, specifically with enable-qt which is not the usual but is required by DQM4HEP.

Here you can find those files.

To install ilcsoft with all needed packages for DQM4HEP, just go to the folder and write:

./ilcsoft-install releases/v01-17-09/release-ilcsoft.cfg

releases/v01-17-09/release-ilcsoft.cfg

releases/v01-17-09/release-versions.py

ilcsoft/qt.py

ilcsoft/root.py

ilcsoft/mysql.py

X-( !! the ilcsoft/root.py is not modified to find the right root version (is in the to-do-list, sorry): I downloaded manuallly so the python script only makes the compilation.

DQMHEP

To install DQM4HEP:

  • First download the git-hub package:

git clone https://github.com/AHCALMonitoring/DQM4HEP DQM4HEP-AHCAL
cd DQM4HEP-AHCAL
  • Then modify the CMakeLists.txt file to point to the AHCALMonitoring git-hub repository

change

SET( xdrstream_repository           "https://github.com/DQM4HEP/xdrstream.git" )
SET( xdrlcio_repository             "https://github.com/DQM4HEP/xdrlcio.git" )
SET( DQMCore_repository             "https://github.com/DQM4HEP/DQMCore.git" )
SET( DQMViz_repository              "https://github.com/DQM4HEP/DQMViz.git" )
SET( DQM4ILC_repository             "https://github.com/AHCALMonitoring/DQM4ILC.git" )
SET( JsonCpp_repository             "https://github.com/DQM4HEP/jsoncpp.git" )
SET( DIM_repository                 "https://github.com/DQM4HEP/dim.git" )
SET( DIMJC_repository               "https://github.com/DQM4HEP/dimjc.git" )
SET( levbdim_repository             "https://github.com/DQM4HEP/levbdim.git" )

by

SET( xdrstream_repository           "https://github.com/AHCALMonitoring/xdrstream.git" )
SET( xdrlcio_repository             "https://github.com/AHCALMonitoring/xdrlcio.git" )
SET( DQMCore_repository             "https://github.com/AHCALMonitoring/DQMCore.git" )
SET( DQMViz_repository              "https://github.com/AHCALMonitoring/DQMViz.git" )
SET( DQM4ILC_repository             "https://github.com/AHCALMonitoring/DQM4ILC.git" )
SET( JsonCpp_repository             "https://github.com/AHCALMonitoring/jsoncpp.git" )
SET( DIM_repository                 "https://github.com/AHCALMonitoring/dim.git" )
SET( DIMJC_repository               "https://github.com/AHCALMonitoring/dimjc.git" )
SET( levbdim_repository             "https://github.com/AHCALMonitoring/levbdim.git" )
  • Finally, modify the paths of following script and install all the packages by running it in the folder that is your DQM4HEP-AHCAL

export LCIO_DIR=/home/calice/Programs/ilcinstall-v01-17-09/v01-17-09/lcio/v02-05
export ILCUTIL_DIR=/home/calice/Programs/ilcinstall-v01-17-09/v01-17-09/ilcutil/v01-02-01
source /home/calice/Programs/ilcinstall-v01-17-09/v01-17-09/root/5.34.30/bin/thisroot.sh
source /home/calice/Programs/ilcinstall-v01-17-09/v01-17-09/init_ilcsoft.sh
mkdir build
cd build
cmake -DBUILD_DQMVIZ=ON -DBUILD_DQM4ILC=ON -DBUILD_EXAMPLES=ON -DINSTALL_DOC=ON -DDIM_GUI=ON ..
make

You might need to modify the FindLCIO.cmake file on ilcutils. Here is an example (change tha path in the right places) FindLCIO.cmake

Quasi online AHCAL Raw Monitoring

Setting up the monitor

Environment

The first step is to set up the correct environment, since each element should be run in a separate terminal window. Add the following lines to your .bashrc, changing the directories as necessary. Note that this should also be added to the .bashrc for the super user, as one step requires a super user environment.

source /home/calice/Programs/ilcinstall-v01-17-09/v01-17-09/init_ilcsoft.sh
export LD_LIBRARY_PATH=/home/calice/Programs/DQM4HEP-TBMay2016/lib:$LD_LIBRARY_PATH
export PATH=/home/calice/Programs/DQM4HEP-TBMay2016/bin:$PATH
export DIM_DNS_NODE=localhost
export DQM4HEP_PLUGIN_DLL=/home/calice/Programs/DQM4HEP-TBMay2016/lib/libDQM4ILC.so:/home/calice/Programs/DQM4HEP-TBMay2016/lib/libAHCALRawModule.so

Mounting Data

To access recently-taken and live data, the folder of the EUDAQ PC where the LCIO data is written to must be made available by mounting. Enter the following in your terminal:

sshfs calice@131.169.184.85:/home/calice/EUDAQ1.6-TestBeamMay2016/data /home/calice/DataMount -oauto_cache,reconnect,allow_other

DNS and DIMJC Daemon

In a new terminal window with init.sh already executed, enter the following in the terminal window:

dns &
dimjc_daemon

Leave this terminal open for the entire duration; closing it will break the connections between all other processes.

Run Control

In a new terminal window with init.sh already executed, enter this command to start the run control:

dqm4hep_start_run_control_server -r AHCALRunControl -k pass &

The argument after -r is the name of this process. This must be the same as in your XML file. The argument after -k is the password needed to begin or end a run.

Then you need to start the interface to the server. Enter this command:

dqm4hep_start_run_control_interface -r AHCALRunControl

The argument after -r is the name of this process. This must be the same as in your XML file, as well as the server above. A window with a GUI will open. As far as I know, all that is necessary for online monitoring is to press the START RUN button, which ensures that the event collector and analysis module work correctly.

DQM Monitor GUI

In a new terminal window with init.sh already executed, enter this command to start the monitor GUI:

dqm4hep_start_dqm_monitor_gui

This will open a GUI for viewing plots and histograms. To see your histograms, click "Browse", then choose the name of your monitor element collector from the drop-down list. You can select which plots to add using the checkboxes. Clicking "Append" adds all checked plots to the main window. There may be no options displayed if the monitor element collector is not running..

Plots will not be updated or kept unless they are subscribed. Individual plots can be subscribed to by checking the box by their names. All plots can be subscribed to by right-clicking anywhere on the list of plots and selecting "Subscribe all". Once subscribed, monitor elements are stored regardless of whether the plot is displayed. A plot can be displayed by double-clicking it. You can click "Update" to manually refresh all plots, or click "Start Update" to make them update automatically. New canvases for plots can be opened using the + sign at the top-right of the window.

Event Collector

In a new terminal window with init.sh already executed, enter this command to start the event collector:

dqm4hep_start_event_collector -c ahcal_event_collector -s LCIOStreamer

The argument after -c is the name of this process, the name after -s is the name of the file streamer to save events from. These must be the same as in your XML file.

Monitor Element Collector

In a new terminal window with init.sh already executed, enter this command to start the monitor element collector:

dqm4hep_start_monitor_element_collector -c ahcal_me_collector

The argument after -c is the name of this process. This must be the same as in your XML file.

Analysis Module

In a new terminal window with init.sh already executed, enter this command to start the analysis module:

dqm4hep_start_analysis_module -f /home/calice/Programs/DQM4HEP-TBMay2016/DQM4ILC/source/examples/AHCALRawModule/lcioAHCALTests.xml

The argument after -f is the path to the XML file used to define your histograms and other processes. ""IMPORTANT:"" Make sure that all the process names in the XML file are the same as the ones you use at runtime. Ensure that you are using the same type of histogram in the XML file as your module file, otherwise the analysis module will throw segmentation faults as soon as it tries to push data into histograms.

LCIO File Service

This should be the last thing to be run, as when the LCIO file service is started the run will begin and data will start being processed by the analysis module. In a new terminal window with init.sh already executed, enter this command to start the LCIO file service:

dqm4hep_start_lcio_file_service -f /home/calice/DataMount/tests/Run_41750__whatever.slcio -c ahcal_event_collector -t 0

The argument after -f is the path to the SLCIO file to read events from (separate several with a colon), the argument after -c is the name of the event collector to send events to, and the argument after -t is the time interval between reading events and sending them to the event collector (in usec).

Shifter instructions

Asumming that we want to run only one Analysis Module (i.e. Power Pulsing module) and the dns and dimjc_daemon are running in the background (see explanations above).

1) In /home/calice, load all the dqm4hep elements:

source manual_start.sh

This is only needed once, but eventually an element will crash. In this case kill them, for that:

source manual_kill.sh

2) In the Run Control, click Start run and introduce the password defined in the manual_start.sh script, (normally will be "pass")

This is only required once at the begining.

3) Modify accordingly the xml file (for example chosing the channel numbers or the memory cell to use in the plots) and start the analisys module:

dqm4hep_start_analysis_module -f /home/calice/Programs/DQM4HEP-TBMay2016/DQM4ILC/source/examples/AHCALRawModule/TestbeamMay2016_PowePulsing/lcioTestbeamPowerPulsing.xml

4) In the DQM Monitoring window browse for the plots: chose ahcal_me_collector and clic update list. Then append all the plots. All of them will appear in a browser tree in the DQM Monitoring window, suscribe to all of them and organize them in the canvas as you want.

5) Start the streamer with the name of the file to look into (and the full path if you are not in the right path)

dqm4hep_start_lcio_file_service -c ahcal_event_collector -t 0 -f /home/calice/DataMount/tests/Run_41750__whatever.slcio

After the first time that you do 1->5, you only need to repeat, for every run, the steps 3 and 5 (not the 4!!)

instructions for July/August Testbeam

Initial steps

1)

Start/stop everything:

cd /home/calice/TB2016/scripts source

manual_kill.sh

source manual_start.sh

2)

Then, import, in the monitoring tab, the appropiate configuration file, for example MIPcalibration_thirdRound.xml (in scripts folder) that contains all the histograms and canvas already booked and prepared.

3)

Import the setting for the run control (now the channel choice is done in the run control)

i.e. RunControl_asic2.xml

The run control asks for password: just click okay.

4)

Load analysis modules in two terminals

dqm4hep_start_analysis_module -f /home/calice/TB2016/DQM4HEP-AHCAL/DQM4ILC/source/examples/AHCALRawModule/TestbeamJuly2016_Hitmaps/lcioHitMap.xml

dqm4hep_start_analysis_module -f /home/calice/TB2016/DQM4HEP-AHCAL/DQM4ILC/source/examples/AHCALRawModule/TestbeamJuly2016_15channels/lcio15channelsX4.xml

5)

Stream the lcio files

dqm4hep_start_lcio_file_service -v INFO -t 1 -c ahcal_event_collector -f /home/calice/TB2016/Data_July2016/MIP/20160729/Run_4331029p07p201616p51p40.slcio

---

For every new position, we only need to: stop the run, import the run configuration, change channels numbers in the run control, start the run and stream the lcio file.

If we are interested in hit maps, only start stop is needed (not the loading of the conf)

AHCALMonitoring_DQM4HEP (last edited 2016-12-20 10:37:05 by EldwanBrianne)