Differences between revisions 1 and 2
Revision 1 as of 2017-09-15 14:36:43
Size: 16449
Editor: KatjaKrueger
Comment:
Revision 2 as of 2017-09-15 14:49:27
Size: 11897
Editor: KatjaKrueger
Comment:
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
== DQM4HEP Monitoring ==

for help about installing DQM4HEP and preparing the analysis modules, see the [[http://flcwiki.desy.de/AHCALMonitoring_DQM4HEP|| AHCAL DQM4HEP wiki page]]

=== Shifter instructions ===
1) In /home/calice/TB2017/scripts, load all the dqm4hep elements:

{{{
source manual_start.sh
}}}
This is only needed once, but eventually an element will crash. In this case kill them, for that:

{{{
source manual_kill.sh
}}}
2) In the Run Control, click Start run click on okay, no password is needed.
This is only required once at the beginning.

3) in /home/calice/TB2017/DQM4HEP/build start the streamer with the name of the file to look into (and the full path if you are not in the right path), e.g.

{{{
dqm4hep_start_lcio_file_service -c ahcal_event_collector -t 1 -f /home/calice/mounted_data_directory/path_to_file/ROCEvents051081_tp.slcio
}}}

4) if you want to clear the histograms before you look at a new run, you need to stop the run in the run control, wait a bit, and then start the run again. After clicking on update in the DQM Monitoring Window, the histograms should be empty
Line 34: Line 8:
''' copy from 2015, to be checked!!! Katja ''' ==== Tunnel to calice DB at DESY ====
At the moment the monitoring is set up to connect to the database at DESY (to make sure that this is the case check /home/calice/Desktop/AHCAL-Reconstruction/xml/Reco.xml, where you should see the following lines:
{{{
    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
}}}
(using port 33306 on 127.0.0.1)

To set up the IP tunnel to DESY, use the following command (with your user name, on the machine where the online monitoring runs):
{{{ssh -L 33306:flccaldb02:3306 user@bastion.desy.de
}}}
Line 50: Line 34:
[[attachment:QtReco.png|{{attachment:QtReco.png|QtReco|height="200"}}]]
Line 61: Line 44:
(need to be edited for each new Run) (Run number needs to be edited for each new Run, the rest should stay the same)
Line 73: Line 56:
files can be mounted via commands
{{{
sshfs calice@192.168.1.11://F:/ /home/calice/Desktop/DAQ_PC/ -oauto_cache,reconnect,allow_other
sshfs calice@192.168.1.10:/home/calice/ /home/calice/Desktop/EUDAQ_PC/ -oauto_cache,reconnect,allow_other
}}}
Mounts shown by the mount command should look like:
{{{
calice@192.168.1.10:/home/calice/ on /home/calice/Desktop/EUDAQ_PC type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,allow_other)
calice@192.168.1.11://F:/ on /home/calice/Desktop/DAQ_PC type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,allow_other)
}}}
Line 98: Line 71:
[[attachment:OnlineMonitor.png|{{attachment:OnlineMonitor.png|OnlineMonitor|height="200"}}]]
Line 152: Line 124:
/!\ The calice database is not accessible from outside the DESY network, so for CERN, a local database needs to be setup or slcio made.

The latter option is chosen and also the database slcio file are in {{{ ~/Desktop/TB_Soft/Local_db/ }}}. The database setup and making of slcio file is rather difficult so it is reserved for experts (contact Eldwan ( eldwan.brianne@NOSPAMdesy.de ) or Lan ( huong.lan.tran@NOSPAMdesy.de )).
Line 165: Line 133:
[[attachment:Database.png|{{attachment:Database.png|Calice Database|height="200"}}]]

=== Database constants (Local) ===

If you are at DESY, you can directly connect to the database no need to have local files. (Same steering part of GeoCondition Processor as offline reconstruction)

=== Database constants (DESY) ===
Line 173: Line 138:
<parameter name="DBInit" type="string" value="flccaldb02.desy.de:calice:caliceon:Delice.1:3306"/> <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
Line 175: Line 140:
    Ahc2ModuleDescription /cd_calice_Ahc2/TestbeamMay2017/ModuleDescription HEAD
    Ahc2ModuleConnection /cd_calice_Ahc2/TestbeamMay2017/ModuleConnection HEAD
    Ahc2ModuleLocationReference /cd_calice_Ahc2/TestbeamMay2017/ModuleLocationReference HEAD
    Ahc2DetectorTransformation /cd_calice_Ahc2/TestbeamMay2017/DetectorTransformation HEAD
    Ahc2HardwareConnection /cd_calice_Ahc2/TestbeamMay2017/Ahc2HardwareConnection HEAD
    E4DPedestal /cd_calice_Ahc2/TestbeamMay2017/Pedestal HEAD
    E4DGainConstants /cd_calice_Ahc2/TestbeamMay2017/gain_constants HEAD
    E4DGainSlopes /cd_calice_Ahc2/TestbeamMay2017/gain_slopes HEAD
    E4DMipConstants /cd_calice_Ahc2/TestbeamMay2017/mip_constants HEAD
    E4DMipSlopes /cd_calice_Ahc2/TestbeamMay2017/mip_slopes HEAD
    E4DDeadCellMap /cd_calice_Ahc2/TestbeamMay2017/DeadCellMap HEAD
    E4DSaturationParameters /cd_calice_Ahc2/TestbeamMay2017/SaturationParameters HEAD
    E4DIntercalibration /cd_calice_Ahc2/TestbeamMay2017/Intercalibration HEAD
    E4DPhysicsCalibIntercalibration /cd_calice_Ahc2/TestbeamMay2017/PhysicsCalibIntercalibration HEAD
    E4DTimeSlopes /cd_calice_Ahc2/TestbeamMay2017/TimeSlopes HEAD
    E4DTimePedestal /cd_calice_Ahc2/TestbeamMay2017/TimePedestal HEAD
    Ahc2ModuleDescription /cd_calice_Ahc2/TestbeamJuly2017/ModuleDescription HEAD
    Ahc2ModuleConnection /cd_calice_Ahc2/TestbeamJuly2017/ModuleConnection HEAD
    Ahc2ModuleLocationReference /cd_calice_Ahc2/TestbeamJuly2017/ModuleLocationReference HEAD
    Ahc2DetectorTransformation /cd_calice_Ahc2/TestbeamJuly2017/DetectorTransformation HEAD
    Ahc2HardwareConnection /cd_calice_Ahc2/TestbeamJuly2017/Ahc2HardwareConnection HEAD
    E4DPedestal /cd_calice_Ahc2/TestbeamJuly2017/Pedestal HEAD
    E4DGainConstants /cd_calice_Ahc2/TestbeamJuly2017/gain_constants HEAD
    E4DGainSlopes /cd_calice_Ahc2/TestbeamJuly2017/gain_slopes HEAD
    E4DMipConstants /cd_calice_Ahc2/TestbeamJuly2017/mip_constants HEAD
    E4DMipSlopes /cd_calice_Ahc2/TestbeamJuly2017/mip_slopes HEAD
    E4DDeadCellMap /cd_calice_Ahc2/TestbeamJuly2017/DeadCellMap HEAD
    E4DSaturationParameters /cd_calice_Ahc2/TestbeamJuly2017/SaturationParameters HEAD
    E4DIntercalibration /cd_calice_Ahc2/TestbeamJuly2017/Intercalibration HEAD
    E4DPhysicsCalibIntercalibration /cd_calice_Ahc2/TestbeamJuly2017/PhysicsCalibIntercalibration HEAD
    E4DTimeSlopes /cd_calice_Ahc2/TestbeamJuly2017/TimeSlopes HEAD
    E4DTimePedestal /cd_calice_Ahc2/TestbeamJuly2017/TimePedestal HEAD
Line 195: Line 160:
In case you are outside DESY, the connection is restricted to specific IP addresses. To avoid this problem, a tunnel must be established or local database files can be specified instead.
==== Tunnel to calice DB at DESY ====
Edit the /home/calice/Desktop/AHCAL-Reconstruction/xml/Reco.xml file and replace flchcaldb02 --> 127.0.0.1 . Port replacement: 3306 --> 33306
{{{
 <!--parameter name="DBInit" type="string" value="flccaldb02.desy.de:calice:caliceon:Delice.1:3306"/-->
    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
}}}

To make the IP tunnel:
{{{ssh -L 33306:flccaldb02:3306 user@bastion.desy.de
}}}

==== Local database extraction ====
First to create a database file, you need the package lccd (http://ilcsoft.desy.de/portal/software_packages/lccd/index_eng.html). Inside the lccd folder, create a folder build. Then go inside and type :
{{{
cmake $ILCSOFT/ILCSoft.cmake ..
make install
}}}

It will compile the package.

To create a local file, you have to do :
{{{
export LCCD_DB_INIT=flccaldb02.desy.de:calice:caliceon:Delice.1:3306
./build/bin/createdbfile <dbfolder> <tag>
}}}

This will then create a slcio file condDB_<Collection>_<tag>_<querytime>.slcio

To make use of theses file locally then you need to specify in the reconstruction steering file :
This specify the name of the output collection, the path of the slcio file and the name of the collection inside the file.

{{{
<processor name="GeoConditions" type="ConditionsProcessor">
<parameter name="DBFileHandler" type="StringVec">
        Ahc2ModuleDescription <filepath> ModuleDescription
        Ahc2ModuleConnection <filepath> ModuleConnection
        Ahc2ModuleLocationReference <filepath> ModuleLocationReference
        Ahc2DetectorTransformation <filepath> DetectorTransformation
        Ahc2HardwareConnection <filepath> Ahc2HardwareConnection
        E4DPedestal <filepath> Pedestal
        E4DGainConstants <filepath> gain_constants
        E4DGainSlopes <filepath> gain_slopes
        E4DMipConstants <filepath> mip_constants
        E4DMipSlopes <filepath> mip_slopes
        E4DDeadCellMap <filepath> DeadCellMap
        E4DSaturationParameters <filepath> SaturationParameters
        E4DIntercalibration <filepath> Intercalibration
</parameter>
</processor>
}}}
Line 248: Line 162:

'''Kataj: please ignore for the moment, needs to be checked!'''

Online Monitoring and reconstruction for AHCAL Test Beam Campaign at CERN2017

This page is a branch of the main AHCALandHGCALTestBeamCERN2017 page.

Contents

Testbeam Monitoring

Tunnel to calice DB at DESY

At the moment the monitoring is set up to connect to the database at DESY (to make sure that this is the case check /home/calice/Desktop/AHCAL-Reconstruction/xml/Reco.xml, where you should see the following lines:

    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>

(using port 33306 on 127.0.0.1)

To set up the IP tunnel to DESY, use the following command (with your user name, on the machine where the online monitoring runs): {{{ssh -L 33306:flccaldb02:3306 user@bastion.desy.de }}}

The programs are in

/home/calice/Desktop/AHCAL-Reconstruction/bin

We reconstruct and apply the available calibrations to the raw events:

QtReco (reconstruction):

1. Run it by doing

./QtReco_interface

2. Edit the steering file in the folder /home/calice/Desktop/AHCAL-Reconstruction/xml (steering.xml)

Run number
Input_PATH
Output_PATH
mapping

(Run number needs to be edited for each new Run, the rest should stay the same)

3. Then Press Configure -> Check if the lcio file is found.

4. Then Press Start, it will run the reco and the analysis.

5. Wait until it finishes. It should show Analysis Module : destroyed

Now we are ready to show the plots or event displays.

Online monitor for events after reconstruction

OnlineMonitor

It is in:

/home/calice/Desktop/AHCAL-Reconstruction/bin

1. Run it by doing

./QtReco_Client_interface

/!\ QtReco-interface must be opened!

2. Click on connect then enter the Run number with a leading zero!. /!\ Check that it says connected.

3. Then choose what plots you want to look at. /!\ Careful when analysing the plots! Mis-calibrations!

Important (and implemented) distributions

Several distributions are available after analysis :

  • Hit multiplicity
  • Energy sum and spectra per layer
  • Shower profile, beam profile
  • HitMap

  • Temperature (only for raw txt!)

Hit Multiplicity

Energy

Shower profile, beam profile

Muons +120 GeV, tungsten absorber

Pions +50 GeV, tungsten absorber

Electrons +20 GeV, tungsten absorber

HitMap

You can see some reference hitmaps from August 2015 data taking period.

For Muons, +120 GeV, tungsten absorber.

For Pions +50 GeV, tungsten absorber.

For Electrons +20 GeV, tungsten absorber.

Calibration procedure

The QtReco program make use of the Marlin framework and the calice processors in order to reconstruct the raw data (hardware information) to physics data. The reconstruction is done in several steps and need to have access to the database containing mapping of the detector and calibration constants for each channel (gain, Pedestal, MIP).

The reconstruction is done in 4 steps :

  • Event Building from raw txt or raw slcio eudaq files
  • Geometry mapping of the detector
  • Get Calibration constant from database/files
  • Apply Calibration

This is done for each hits in each events.

Database constants (DESY)

<processor name="GeoConditions" type="ConditionsProcessor">
<parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
 <parameter name="DBCondHandler" type="StringVec">
    Ahc2ModuleDescription        /cd_calice_Ahc2/TestbeamJuly2017/ModuleDescription               HEAD
    Ahc2ModuleConnection         /cd_calice_Ahc2/TestbeamJuly2017/ModuleConnection                HEAD
    Ahc2ModuleLocationReference  /cd_calice_Ahc2/TestbeamJuly2017/ModuleLocationReference         HEAD
    Ahc2DetectorTransformation   /cd_calice_Ahc2/TestbeamJuly2017/DetectorTransformation          HEAD
    Ahc2HardwareConnection       /cd_calice_Ahc2/TestbeamJuly2017/Ahc2HardwareConnection          HEAD
    E4DPedestal                  /cd_calice_Ahc2/TestbeamJuly2017/Pedestal                        HEAD
    E4DGainConstants             /cd_calice_Ahc2/TestbeamJuly2017/gain_constants                  HEAD
    E4DGainSlopes                /cd_calice_Ahc2/TestbeamJuly2017/gain_slopes                     HEAD
    E4DMipConstants              /cd_calice_Ahc2/TestbeamJuly2017/mip_constants                   HEAD
    E4DMipSlopes                 /cd_calice_Ahc2/TestbeamJuly2017/mip_slopes                      HEAD
    E4DDeadCellMap               /cd_calice_Ahc2/TestbeamJuly2017/DeadCellMap                     HEAD
    E4DSaturationParameters      /cd_calice_Ahc2/TestbeamJuly2017/SaturationParameters            HEAD
    E4DIntercalibration          /cd_calice_Ahc2/TestbeamJuly2017/Intercalibration                HEAD
    E4DPhysicsCalibIntercalibration          /cd_calice_Ahc2/TestbeamJuly2017/PhysicsCalibIntercalibration                HEAD
    E4DTimeSlopes                /cd_calice_Ahc2/TestbeamJuly2017/TimeSlopes                      HEAD
    E4DTimePedestal              /cd_calice_Ahc2/TestbeamJuly2017/TimePedestal                    HEAD
    </parameter>
</processor>

Event display

Kataj: please ignore for the moment, needs to be checked!

After applying the reconstruction and the corrections, we also can have a look on the aspect of the events in our detector. Actually, we can do it in parallel to the reconstruction, since it reads the events in the slcio file created by the QtReco program. For that we use a schematic 3D view of our detector that allows us to nicely and very intuitively discern the structure of the beam (content of pions, muons, electrons, etc), the position of the tracks, etc.

To execute the event display go in :

~/Desktop/Event_Display/

and start

glced&
./myMarlin.sh xml/CED_2017May.xml

To ship the data to the online display from a correct file, xml steering file /home/calice/Desktop/Event_Display/xml/CED_2017May.xml has to be edited. File which will be displayd is here::

<parameter name="LCIOInputFiles">
  <!--/home/calice/Desktop/Reco_Data/BuiltEvents000025_tp.slcio /-->
  /home/calice/Desktop/Reco_Data/ROCEvents050917_tp.slcio
</parameter>

and database connection has to be fixed in a same way as for reconstruction:

<processor name="GeoConditions" type="ConditionsProcessor">
    <!--flccaldb02 -->
    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
    <parameter name="DBCondHandler" type="StringVec"> 
    Ahc2ModuleDescription        /cd_calice_Ahc2/TestbeamMay2017/ModuleDescription               HEAD
    Ahc2ModuleConnection         /cd_calice_Ahc2/TestbeamMay2017/ModuleConnection                HEAD
    Ahc2ModuleLocationReference  /cd_calice_Ahc2/TestbeamMay2017/ModuleLocationReference         HEAD
    Ahc2DetectorTransformation   /cd_calice_Ahc2/TestbeamMay2017/DetectorTransformation          HEAD
    Ahc2HardwareConnection       /cd_calice_Ahc2/TestbeamMay2017/Ahc2HardwareConnection          HEAD
    E4DPedestal                  /cd_calice_Ahc2/TestbeamMay2017/Pedestal                        HEAD
    E4DGainConstants             /cd_calice_Ahc2/TestbeamMay2017/gain_constants                  HEAD
    E4DGainSlopes                /cd_calice_Ahc2/TestbeamMay2017/gain_slopes                     HEAD
    E4DMipConstants              /cd_calice_Ahc2/TestbeamMay2017/mip_constants                   HEAD
    E4DMipSlopes                 /cd_calice_Ahc2/TestbeamMay2017/mip_slopes                      HEAD
    E4DDeadCellMap               /cd_calice_Ahc2/TestbeamMay2017/DeadCellMap                     HEAD
    E4DSaturationParameters      /cd_calice_Ahc2/TestbeamMay2017/SaturationParameters            HEAD
    E4DIntercalibration          /cd_calice_Ahc2/TestbeamMay2017/Intercalibration                HEAD
    E4DPhysicsCalibIntercalibration          /cd_calice_Ahc2/TestbeamMay2017/PhysicsCalibIntercalibration                HEAD
    E4DTimeSlopes                /cd_calice_Ahc2/TestbeamMay2017/TimeSlopes                      HEAD
    E4DTimePedestal              /cd_calice_Ahc2/TestbeamMay2017/TimePedestal                    HEAD
    </parameter> 
  </processor>

The program has a tool to make screenshots. This is found in the "tools" tab and do screenshots in tga format.

Some standard event display are shown in the following pictures.

Electron (-) candidates (10 GeV, steel absorber)

Muon (-) candidate ( steel absorber)

Pion (-) candidate (50 GeV, steel absorber)

Pion (+) candidates (50 GeV, tungsten absorber)

Electron (+) candidates (20 GeV, tungsten absorber)

To make these nice sets of eventdisplays, we can use a few set of "Jiri's" commands:

1) Convert to png, since the saved pictures in the eventdisplay use the tga format

ls *.tga|parallel convert {} ~
{.}.png

2) Create the set, by appending individual pictures:

convert \( glced-1.png glced-2.png +append \) \( glced-3.png glced-4.png +append \) -append test.png

(for 2x2)
or

convert \( glced-1.png glced-2.png glced-3.png +append +append \) \( glced-4.png glced-5.png glced-6.png +append +append \) \( glced-7.png glced-8.png glced-9.png \) -append -append r30079_pick_of_9events.png

(for 3x3)

dCache

The procedure of copy to DCache is described here: http://flcwiki.desy.de/GridInstall

On the DCache computer, a Virtual machine is automatically started with the PC. To invoke the program, following command is needed:

ssh -X -p 2222 calice@localhost "cd dCache-gui; bin/dCache_start_gui"


CategoryHCAL

AHCALandHGCALTestBeamCERN2017/Monitoring (last edited 2017-09-15 15:10:18 by KatjaKrueger)