Differences between revisions 1 and 7 (spanning 6 versions)
Revision 1 as of 2017-09-15 14:36:43
Size: 16449
Editor: KatjaKrueger
Comment:
Revision 7 as of 2017-09-15 15:10:18
Size: 11898
Editor: KatjaKrueger
Comment:
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
== DQM4HEP Monitoring ==

for help about installing DQM4HEP and preparing the analysis modules, see the [[http://flcwiki.desy.de/AHCALMonitoring_DQM4HEP|| AHCAL DQM4HEP wiki page]]

=== Shifter instructions ===
1) In /home/calice/TB2017/scripts, load all the dqm4hep elements:

{{{
source manual_start.sh
}}}
This is only needed once, but eventually an element will crash. In this case kill them, for that:

{{{
source manual_kill.sh
}}}
2) In the Run Control, click Start run click on okay, no password is needed.
This is only required once at the beginning.

3) in /home/calice/TB2017/DQM4HEP/build start the streamer with the name of the file to look into (and the full path if you are not in the right path), e.g.

{{{
dqm4hep_start_lcio_file_service -c ahcal_event_collector -t 1 -f /home/calice/mounted_data_directory/path_to_file/ROCEvents051081_tp.slcio
}}}

4) if you want to clear the histograms before you look at a new run, you need to stop the run in the run control, wait a bit, and then start the run again. After clicking on update in the DQM Monitoring Window, the histograms should be empty
Line 34: Line 8:
''' copy from 2015, to be checked!!! Katja ''' '''runs on flchcallab5'''

==== Tunnel to calice DB at DESY ====
At the moment the monitoring is set up to connect to the database at DESY (to make sure that this is the case check /home/calice/Desktop/AHCAL-Reconstruction/xml/Reco.xml, where you should see the following lines:
{{{
    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
}}}
(using port 33306 on 127.0.0.1)

To set up the IP tunnel to DESY, use the following command (with your user name, on the machine where the online monitoring runs):
{{{
ssh -L 33306:flccaldb02:3306 user@bastion.desy.de
}}}
Line 50: Line 37:
[[attachment:QtReco.png|{{attachment:QtReco.png|QtReco|height="200"}}]]
Line 61: Line 47:
(need to be edited for each new Run) (Run number needs to be edited for each new Run, the rest should stay the same)
Line 73: Line 59:
files can be mounted via commands
{{{
sshfs calice@192.168.1.11://F:/ /home/calice/Desktop/DAQ_PC/ -oauto_cache,reconnect,allow_other
sshfs calice@192.168.1.10:/home/calice/ /home/calice/Desktop/EUDAQ_PC/ -oauto_cache,reconnect,allow_other
}}}
Mounts shown by the mount command should look like:
{{{
calice@192.168.1.10:/home/calice/ on /home/calice/Desktop/EUDAQ_PC type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,allow_other)
calice@192.168.1.11://F:/ on /home/calice/Desktop/DAQ_PC type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,allow_other)
}}}
Line 98: Line 74:
[[attachment:OnlineMonitor.png|{{attachment:OnlineMonitor.png|OnlineMonitor|height="200"}}]]
Line 111: Line 86:
 * HitMap  * Hit Map
Line 114: Line 89:

==== Hit Multiplicity ====
[[attachment:NHits_profiles.png|{{attachment:NHits_profiles.png|NHits Profile|height="250"}}]] [[attachment:NHits_per_layer.png|{{attachment:NHits_per_layer.png|NHits Per Layer|height="250"}}]]

==== Energy ====
[[attachment:Energy_sum.png|{{attachment:Energy_sum.png|Energy Sum|height="250"}}]] [[attachment:Energy_per_layer.png|{{attachment:Energy_per_layer.png|Energy Per Layer|height="250"}}]]
'''left plots are for 300 GeV pions in July 2017, right ones for a muon run in August 2017'''
==== Hit Profiles per layer ====
[[attachment:NHits_profile_pion.png|{{attachment:NHits_profile_pion.png|NHits Profile for Pions|height="400"}}]]
[[attachment:NHits_profile_muon.png|{{attachment:NHits_profile_muon.png|NHits Profile for Muons|height="400"}}]]

==== Number of hits per layer ====
[[attachment:NHits_per_layer_pion.png|{{attachment:NHits_per_layer_pion.png|NHits Per Layer Pions|height="400"}}]]
[[attachment:NHits_per_layer_muon.png|{{attachment:NHits_per_layer_muon.png|NHits Per Layer Muons|height="400"}}]]

==== Total Energy ====
[[attachment:Energy_sum_pion.png|{{attachment:Energy_sum_pion.png|Energy Sum Pions|height="400"}}]]
[[attachment:Energy_sum_muon.png|{{attachment:Energy_sum_muon.png|Energy Sum Muons|height="400"}}]]

==== Hit Energy (of all cells within a layer) ====
[[attachment:HitEnergy_per_layer_pion.png|{{attachment:HitEnergy_per_layer_pion.png|HitEnergy Per Layer Pions|height="400"}}]]
[[attachment:HitEnergy_per_layer_muon.png|{{attachment:HitEnergy_per_layer_muon.png|HitEnergy Per Layer Muons|height="400"}}]]
Line 122: Line 108:
Muons +120 GeV, tungsten absorber

[[attachment:shower_muon120_tungsten.png|{{attachment:shower_muon120_tungsten.png|shower_muon120_tungsten|height="250"}}]]

Pions +50 GeV, tungsten absorber

[[attachment:shower_pion50_tungsten.png|{{attachment:shower_pion50_tungsten.png|shower_pion50_tungsten|height="250"}}]]

Electrons +20 GeV, tungsten absorber

[[attachment:shower_electron20_tungsten.png|{{attachment:shower_electron20_tungsten.png|shower_electro20_tungsten|height="250"}}]]
[[attachment:Shower_pion.png|{{attachment:Shower_pion.png|Shower variables Pions|height="400"}}]]
[[attachment:Shower_muon.png|{{attachment:Shower_muon.png|Shower variables Muons|height="400"}}]]
Line 135: Line 112:
You can see some reference hitmaps from August 2015 data taking period.

For Muons, +120 GeV, tungsten absorber.

[[attachment:muon120GeV_tungsten_hitmap.png|{{attachment:muon120GeV_tungsten_hitmap.png|Hit Map Muon 120 GeV, tungsten absorber|height="250"}}]]

For Pions +50 GeV, tungsten absorber.

[[attachment:pion50GeV_tungsten_hitmap.png|{{attachment:pion50GeV_tungsten_hitmap.png|Hit Map Pion 50 GeV, tungsten absorber|height="250"}}]]

For Electrons +20 GeV, tungsten absorber.

[[attachment:electron20GeV_tungsten_hitmap.png|{{attachment:electron20GeV_tungsten_hitmap.png|Hit Map electron 20 GeV, tungsten absorber|height="250"}}]]
[[attachment:Hitmap_pion.png|{{attachment:Hitmap_pion.png|Hit Map Pions|height="400"}}]]
[[attachment:Hitmap_muon.png|{{attachment:Hitmap_muon.png|Hit Map Muons|height="400"}}]]
Line 151: Line 117:

/!\ The calice database is not accessible from outside the DESY network, so for CERN, a local database needs to be setup or slcio made.

The latter option is chosen and also the database slcio file are in {{{ ~/Desktop/TB_Soft/Local_db/ }}}. The database setup and making of slcio file is rather difficult so it is reserved for experts (contact Eldwan ( eldwan.brianne@NOSPAMdesy.de ) or Lan ( huong.lan.tran@NOSPAMdesy.de )).
Line 165: Line 127:
[[attachment:Database.png|{{attachment:Database.png|Calice Database|height="200"}}]]

=== Database constants (Local) ===

If you are at DESY, you can directly connect to the database no need to have local files. (Same steering part of GeoCondition Processor as offline reconstruction)

=== Database constants (DESY) ===
Line 173: Line 132:
<parameter name="DBInit" type="string" value="flccaldb02.desy.de:calice:caliceon:Delice.1:3306"/> <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
Line 175: Line 134:
    Ahc2ModuleDescription /cd_calice_Ahc2/TestbeamMay2017/ModuleDescription HEAD
    Ahc2ModuleConnection /cd_calice_Ahc2/TestbeamMay2017/ModuleConnection HEAD
    Ahc2ModuleLocationReference /cd_calice_Ahc2/TestbeamMay2017/ModuleLocationReference HEAD
    Ahc2DetectorTransformation /cd_calice_Ahc2/TestbeamMay2017/DetectorTransformation HEAD
    Ahc2HardwareConnection /cd_calice_Ahc2/TestbeamMay2017/Ahc2HardwareConnection HEAD
    E4DPedestal /cd_calice_Ahc2/TestbeamMay2017/Pedestal HEAD
    E4DGainConstants /cd_calice_Ahc2/TestbeamMay2017/gain_constants HEAD
    E4DGainSlopes /cd_calice_Ahc2/TestbeamMay2017/gain_slopes HEAD
    E4DMipConstants /cd_calice_Ahc2/TestbeamMay2017/mip_constants HEAD
    E4DMipSlopes /cd_calice_Ahc2/TestbeamMay2017/mip_slopes HEAD
    E4DDeadCellMap /cd_calice_Ahc2/TestbeamMay2017/DeadCellMap HEAD
    E4DSaturationParameters /cd_calice_Ahc2/TestbeamMay2017/SaturationParameters HEAD
    E4DIntercalibration /cd_calice_Ahc2/TestbeamMay2017/Intercalibration HEAD
    E4DPhysicsCalibIntercalibration /cd_calice_Ahc2/TestbeamMay2017/PhysicsCalibIntercalibration HEAD
    E4DTimeSlopes /cd_calice_Ahc2/TestbeamMay2017/TimeSlopes HEAD
    E4DTimePedestal /cd_calice_Ahc2/TestbeamMay2017/TimePedestal HEAD
    Ahc2ModuleDescription /cd_calice_Ahc2/TestbeamJuly2017/ModuleDescription HEAD
    Ahc2ModuleConnection /cd_calice_Ahc2/TestbeamJuly2017/ModuleConnection HEAD
    Ahc2ModuleLocationReference /cd_calice_Ahc2/TestbeamJuly2017/ModuleLocationReference HEAD
    Ahc2DetectorTransformation /cd_calice_Ahc2/TestbeamJuly2017/DetectorTransformation HEAD
    Ahc2HardwareConnection /cd_calice_Ahc2/TestbeamJuly2017/Ahc2HardwareConnection HEAD
    E4DPedestal /cd_calice_Ahc2/TestbeamJuly2017/Pedestal HEAD
    E4DGainConstants /cd_calice_Ahc2/TestbeamJuly2017/gain_constants HEAD
    E4DGainSlopes /cd_calice_Ahc2/TestbeamJuly2017/gain_slopes HEAD
    E4DMipConstants /cd_calice_Ahc2/TestbeamJuly2017/mip_constants HEAD
    E4DMipSlopes /cd_calice_Ahc2/TestbeamJuly2017/mip_slopes HEAD
    E4DDeadCellMap /cd_calice_Ahc2/TestbeamJuly2017/DeadCellMap HEAD
    E4DSaturationParameters /cd_calice_Ahc2/TestbeamJuly2017/SaturationParameters HEAD
    E4DIntercalibration /cd_calice_Ahc2/TestbeamJuly2017/Intercalibration HEAD
    E4DPhysicsCalibIntercalibration /cd_calice_Ahc2/TestbeamJuly2017/PhysicsCalibIntercalibration HEAD
    E4DTimeSlopes /cd_calice_Ahc2/TestbeamJuly2017/TimeSlopes HEAD
    E4DTimePedestal /cd_calice_Ahc2/TestbeamJuly2017/TimePedestal HEAD
Line 195: Line 154:
In case you are outside DESY, the connection is restricted to specific IP addresses. To avoid this problem, a tunnel must be established or local database files can be specified instead.
==== Tunnel to calice DB at DESY ====
Edit the /home/calice/Desktop/AHCAL-Reconstruction/xml/Reco.xml file and replace flchcaldb02 --> 127.0.0.1 . Port replacement: 3306 --> 33306
{{{
 <!--parameter name="DBInit" type="string" value="flccaldb02.desy.de:calice:caliceon:Delice.1:3306"/-->
    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
}}}

To make the IP tunnel:
{{{ssh -L 33306:flccaldb02:3306 user@bastion.desy.de
}}}

==== Local database extraction ====
First to create a database file, you need the package lccd (http://ilcsoft.desy.de/portal/software_packages/lccd/index_eng.html). Inside the lccd folder, create a folder build. Then go inside and type :
{{{
cmake $ILCSOFT/ILCSoft.cmake ..
make install
}}}

It will compile the package.

To create a local file, you have to do :
{{{
export LCCD_DB_INIT=flccaldb02.desy.de:calice:caliceon:Delice.1:3306
./build/bin/createdbfile <dbfolder> <tag>
}}}

This will then create a slcio file condDB_<Collection>_<tag>_<querytime>.slcio

To make use of theses file locally then you need to specify in the reconstruction steering file :
This specify the name of the output collection, the path of the slcio file and the name of the collection inside the file.

{{{
<processor name="GeoConditions" type="ConditionsProcessor">
<parameter name="DBFileHandler" type="StringVec">
        Ahc2ModuleDescription <filepath> ModuleDescription
        Ahc2ModuleConnection <filepath> ModuleConnection
        Ahc2ModuleLocationReference <filepath> ModuleLocationReference
        Ahc2DetectorTransformation <filepath> DetectorTransformation
        Ahc2HardwareConnection <filepath> Ahc2HardwareConnection
        E4DPedestal <filepath> Pedestal
        E4DGainConstants <filepath> gain_constants
        E4DGainSlopes <filepath> gain_slopes
        E4DMipConstants <filepath> mip_constants
        E4DMipSlopes <filepath> mip_slopes
        E4DDeadCellMap <filepath> DeadCellMap
        E4DSaturationParameters <filepath> SaturationParameters
        E4DIntercalibration <filepath> Intercalibration
</parameter>
</processor>
}}}
Line 248: Line 156:

'''Kataj: please ignore for the moment, needs to be checked!'''

Online Monitoring and reconstruction for AHCAL Test Beam Campaign at CERN2017

This page is a branch of the main AHCALandHGCALTestBeamCERN2017 page.

Contents

Testbeam Monitoring

runs on flchcallab5

Tunnel to calice DB at DESY

At the moment the monitoring is set up to connect to the database at DESY (to make sure that this is the case check /home/calice/Desktop/AHCAL-Reconstruction/xml/Reco.xml, where you should see the following lines:

    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>

(using port 33306 on 127.0.0.1)

To set up the IP tunnel to DESY, use the following command (with your user name, on the machine where the online monitoring runs):

ssh -L 33306:flccaldb02:3306 user@bastion.desy.de

The programs are in

/home/calice/Desktop/AHCAL-Reconstruction/bin

We reconstruct and apply the available calibrations to the raw events:

QtReco (reconstruction):

1. Run it by doing

./QtReco_interface

2. Edit the steering file in the folder /home/calice/Desktop/AHCAL-Reconstruction/xml (steering.xml)

Run number
Input_PATH
Output_PATH
mapping

(Run number needs to be edited for each new Run, the rest should stay the same)

3. Then Press Configure -> Check if the lcio file is found.

4. Then Press Start, it will run the reco and the analysis.

5. Wait until it finishes. It should show Analysis Module : destroyed

Now we are ready to show the plots or event displays.

Online monitor for events after reconstruction

OnlineMonitor

It is in:

/home/calice/Desktop/AHCAL-Reconstruction/bin

1. Run it by doing

./QtReco_Client_interface

/!\ QtReco-interface must be opened!

2. Click on connect then enter the Run number with a leading zero!. /!\ Check that it says connected.

3. Then choose what plots you want to look at. /!\ Careful when analysing the plots! Mis-calibrations!

Important (and implemented) distributions

Several distributions are available after analysis :

  • Hit multiplicity
  • Energy sum and spectra per layer
  • Shower profile, beam profile
  • Hit Map
  • Temperature (only for raw txt!)

left plots are for 300 GeV pions in July 2017, right ones for a muon run in August 2017

Hit Profiles per layer

NHits Profile for Pions NHits Profile for Muons

Number of hits per layer

NHits Per Layer Pions NHits Per Layer Muons

Total Energy

Energy Sum Pions Energy Sum Muons

Hit Energy (of all cells within a layer)

HitEnergy Per Layer Pions HitEnergy Per Layer Muons

Shower profile, beam profile

Shower variables Pions Shower variables Muons

HitMap

Hit Map Pions Hit Map Muons

Calibration procedure

The QtReco program make use of the Marlin framework and the calice processors in order to reconstruct the raw data (hardware information) to physics data. The reconstruction is done in several steps and need to have access to the database containing mapping of the detector and calibration constants for each channel (gain, Pedestal, MIP).

The reconstruction is done in 4 steps :

  • Event Building from raw txt or raw slcio eudaq files
  • Geometry mapping of the detector
  • Get Calibration constant from database/files
  • Apply Calibration

This is done for each hits in each events.

Database constants (DESY)

<processor name="GeoConditions" type="ConditionsProcessor">
<parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
 <parameter name="DBCondHandler" type="StringVec">
    Ahc2ModuleDescription        /cd_calice_Ahc2/TestbeamJuly2017/ModuleDescription               HEAD
    Ahc2ModuleConnection         /cd_calice_Ahc2/TestbeamJuly2017/ModuleConnection                HEAD
    Ahc2ModuleLocationReference  /cd_calice_Ahc2/TestbeamJuly2017/ModuleLocationReference         HEAD
    Ahc2DetectorTransformation   /cd_calice_Ahc2/TestbeamJuly2017/DetectorTransformation          HEAD
    Ahc2HardwareConnection       /cd_calice_Ahc2/TestbeamJuly2017/Ahc2HardwareConnection          HEAD
    E4DPedestal                  /cd_calice_Ahc2/TestbeamJuly2017/Pedestal                        HEAD
    E4DGainConstants             /cd_calice_Ahc2/TestbeamJuly2017/gain_constants                  HEAD
    E4DGainSlopes                /cd_calice_Ahc2/TestbeamJuly2017/gain_slopes                     HEAD
    E4DMipConstants              /cd_calice_Ahc2/TestbeamJuly2017/mip_constants                   HEAD
    E4DMipSlopes                 /cd_calice_Ahc2/TestbeamJuly2017/mip_slopes                      HEAD
    E4DDeadCellMap               /cd_calice_Ahc2/TestbeamJuly2017/DeadCellMap                     HEAD
    E4DSaturationParameters      /cd_calice_Ahc2/TestbeamJuly2017/SaturationParameters            HEAD
    E4DIntercalibration          /cd_calice_Ahc2/TestbeamJuly2017/Intercalibration                HEAD
    E4DPhysicsCalibIntercalibration          /cd_calice_Ahc2/TestbeamJuly2017/PhysicsCalibIntercalibration                HEAD
    E4DTimeSlopes                /cd_calice_Ahc2/TestbeamJuly2017/TimeSlopes                      HEAD
    E4DTimePedestal              /cd_calice_Ahc2/TestbeamJuly2017/TimePedestal                    HEAD
    </parameter>
</processor>

Event display

Kataj: please ignore for the moment, needs to be checked!

After applying the reconstruction and the corrections, we also can have a look on the aspect of the events in our detector. Actually, we can do it in parallel to the reconstruction, since it reads the events in the slcio file created by the QtReco program. For that we use a schematic 3D view of our detector that allows us to nicely and very intuitively discern the structure of the beam (content of pions, muons, electrons, etc), the position of the tracks, etc.

To execute the event display go in :

~/Desktop/Event_Display/

and start

glced&
./myMarlin.sh xml/CED_2017May.xml

To ship the data to the online display from a correct file, xml steering file /home/calice/Desktop/Event_Display/xml/CED_2017May.xml has to be edited. File which will be displayd is here::

<parameter name="LCIOInputFiles">
  <!--/home/calice/Desktop/Reco_Data/BuiltEvents000025_tp.slcio /-->
  /home/calice/Desktop/Reco_Data/ROCEvents050917_tp.slcio
</parameter>

and database connection has to be fixed in a same way as for reconstruction:

<processor name="GeoConditions" type="ConditionsProcessor">
    <!--flccaldb02 -->
    <parameter name="DBInit" type="string" value="127.0.0.1:calice:caliceon:Delice.1:33306"/>
    <parameter name="DBCondHandler" type="StringVec"> 
    Ahc2ModuleDescription        /cd_calice_Ahc2/TestbeamMay2017/ModuleDescription               HEAD
    Ahc2ModuleConnection         /cd_calice_Ahc2/TestbeamMay2017/ModuleConnection                HEAD
    Ahc2ModuleLocationReference  /cd_calice_Ahc2/TestbeamMay2017/ModuleLocationReference         HEAD
    Ahc2DetectorTransformation   /cd_calice_Ahc2/TestbeamMay2017/DetectorTransformation          HEAD
    Ahc2HardwareConnection       /cd_calice_Ahc2/TestbeamMay2017/Ahc2HardwareConnection          HEAD
    E4DPedestal                  /cd_calice_Ahc2/TestbeamMay2017/Pedestal                        HEAD
    E4DGainConstants             /cd_calice_Ahc2/TestbeamMay2017/gain_constants                  HEAD
    E4DGainSlopes                /cd_calice_Ahc2/TestbeamMay2017/gain_slopes                     HEAD
    E4DMipConstants              /cd_calice_Ahc2/TestbeamMay2017/mip_constants                   HEAD
    E4DMipSlopes                 /cd_calice_Ahc2/TestbeamMay2017/mip_slopes                      HEAD
    E4DDeadCellMap               /cd_calice_Ahc2/TestbeamMay2017/DeadCellMap                     HEAD
    E4DSaturationParameters      /cd_calice_Ahc2/TestbeamMay2017/SaturationParameters            HEAD
    E4DIntercalibration          /cd_calice_Ahc2/TestbeamMay2017/Intercalibration                HEAD
    E4DPhysicsCalibIntercalibration          /cd_calice_Ahc2/TestbeamMay2017/PhysicsCalibIntercalibration                HEAD
    E4DTimeSlopes                /cd_calice_Ahc2/TestbeamMay2017/TimeSlopes                      HEAD
    E4DTimePedestal              /cd_calice_Ahc2/TestbeamMay2017/TimePedestal                    HEAD
    </parameter> 
  </processor>

The program has a tool to make screenshots. This is found in the "tools" tab and do screenshots in tga format.

Some standard event display are shown in the following pictures.

Electron (-) candidates (10 GeV, steel absorber)

Muon (-) candidate ( steel absorber)

Pion (-) candidate (50 GeV, steel absorber)

Pion (+) candidates (50 GeV, tungsten absorber)

Electron (+) candidates (20 GeV, tungsten absorber)

To make these nice sets of eventdisplays, we can use a few set of "Jiri's" commands:

1) Convert to png, since the saved pictures in the eventdisplay use the tga format

ls *.tga|parallel convert {} ~
{.}.png

2) Create the set, by appending individual pictures:

convert \( glced-1.png glced-2.png +append \) \( glced-3.png glced-4.png +append \) -append test.png

(for 2x2)
or

convert \( glced-1.png glced-2.png glced-3.png +append +append \) \( glced-4.png glced-5.png glced-6.png +append +append \) \( glced-7.png glced-8.png glced-9.png \) -append -append r30079_pick_of_9events.png

(for 3x3)

dCache

The procedure of copy to DCache is described here: http://flcwiki.desy.de/GridInstall

On the DCache computer, a Virtual machine is automatically started with the PC. To invoke the program, following command is needed:

ssh -X -p 2222 calice@localhost "cd dCache-gui; bin/dCache_start_gui"


CategoryHCAL

AHCALandHGCALTestBeamCERN2017/Monitoring (last edited 2017-09-15 15:10:18 by KatjaKrueger)