This page is dedicated to the general software that we use for the current engineering prototype. The software is divided in two categories:

Standalone Software: Gain Calibration and Pedestal Offset Corrections

The code is divided in three main parts: a text to root file converter, the pedestal offset extractor and the gain calibrator. Is important to notice that this code is though to study only standard LED calibration runs taken in Lab mode (forced trigger).

/!\ Theses standalone packages are being ported to Marlin as processors in calice_calib, please visit this section for more information, and the standalone repository might not follow all the changes included in this repository. Use it with careful.

This section includes aonly a brief description of each piece of software, please visit this wiki for more detailed information.

Convert root (text to root file converter)

The data is saved, by the Labview programs that run the electronic setup, in ASCII format. This part of the code converts these files to more manageable root files.

This software is used to convert LED txt files to rootfiles (no event building). The rootfile will contain all the Raw data from LED runs. It runs simply by doing :

$ ./convert_root [INPUT FOLDER] [OUTPUT FOLDER]

The software will automatically list all txt files in the input folder and create the rootfile in the output folder. The format of the data in the txt file should be :

sscanf(line.c_str(), "%i %i %i %i %i %i %i %i %i", &CycleNr, &BunchXID, &ChipID2, &EvtNr, &chn, &TDC, &ADC, &Hit_Bit, &Gain_Bit); for HDMI DAQ
sscanf(line.c_str(), "%i %i %i %i %i %i %i %i %i %i %i %i", &BunchXID, &CycleNr, &ChipID2, &ASICNr, &EvtNr, &chn, &TDC, &ADC, &xPos, &yPos, &Hit_Bit, &Gain_Bit); for USB DAQ

Pedestal Calibration

This software is used to do the pedestal calibration for LED runs Lab Mode ! It takes a pedestal data file (rootfile) as input and provide a .tsv file as output containing the Mean/RMS pedestal with all memory cells (from 1 to 16) as also the relative shift of memory cells compared to the 2nd memory, starting from 1, not from 0. For the data obtained with the USB DAQ, the memory cells order is inverted, so the offset is calculated with respect memory cell 15 (the previous to the last one).

Here is an example of the output file :

#pedestal positions & memory cell dependent offsets from file "/nfs/dust/ilc/user/ebrianne/AHCAL_TestbeamJuly2015/LED/20150704/RootFiles/Pedestal_1000cycles_0000mV__04p07p2015__19o24o21.root"
#chip   chn     pedposall       pedwidthall     pedposcell1     pedposcell2     pedposcell3     ...     pedposcell16
129     0       254.502 7.33865 251.683 0       -5.19458        -2.60246        4.49419 -2.7122 -4.45008        -11.4947        -4.03073        4.35523 -4.66934        -4.62959        -2.02395      1.58653 -7.16559        -2.51706
129     1       267.647 8.0676  266.052 0       -9.12801        -4.98149        -2.4989 -2.80231        -7.32887        -4.94836        -4.0798 5.92922 -0.981946       0.370887        0.366007      8.05469 5.36936 -1.85978

/!\ The file header says "pedposcellX" when it really means "pedestal offset for memory cell X" defined as: pedposcell2 - pedposcellX

Gain Calibration

This software is used to do the Gain calibration of the AHCAL EPT. It takes rootfiles of the standard LED runs as input and the pedestal file (.tsv) in order to produce ADC Spectrums for each channels. Then it performs a MultiGaussian Fit on the Spectra. The Fit is in the form :

mg += par[2*i+3]*TMath::Gaus(xx,gain*i+p0,par[2*i+4]);

par[2*i+3] is the normalisation of the peak i
gain is the distance between peaks -> knowledge input there (as we know that it should be like that)
p0 is the 1st peak position (pedestal)
par[2*i+4] is the width of the peak at position i

It also performs a FFT fit in order to get a comparison with another method. At the end, one can obtain .tsv files for each LED Voltages containing the gain from MultiGaussianFit, the gain from the FFT analysis and the gain error for each channel fitted.


Calice Software

Attachements : Reco.xml myMarlin CED.xml

How to compile the Calice software

You need to have a NAF account to have access to the data and the software source files. Then, the software is available on the calice svn : It contains different folders:

You can download them via svn by this command : svn co<package_name>/trunk/ <target_folder>

Preparing the environment

You need to log to one of the NAF machine by typing

$ ssh -XY

and svn checkout to make a local copy on your computing environment

Before building anything you need to set the environment (ILCSOFT, ROOT..).

$ source /afs/

Create a bash to make this easier, then just source it.

/!\ Note : If you want to install Calice soft and run Marlin on Ubuntu 16.04. You will need to uninstall the version of mysql present and compile mysql-5.1. /!\

See here :

tar xzvf mysql-5.1.73.tar.gz
cd mysql-5.1.73

./configure \
'--prefix=/usr' \
'--exec-prefix=/usr' \
'--libexecdir=/usr/sbin' \
'--datadir=/usr/share' \
'--localstatedir=/var/lib/mysql' \
'--includedir=/usr/include' \
'--infodir=/usr/share/info' \
'--mandir=/usr/share/man' \
'--with-system-type=debian-linux-gnu' \
'--enable-shared' \
'--enable-static' \
'--enable-thread-safe-client' \
'--enable-assembler' \
'--enable-local-infile' \
'--with-fast-mutexes' \
'--with-big-tables' \
'--with-unix-socket-path=/var/run/mysqld/mysqld.sock' \
'--with-mysqld-user=mysql' \
'--with-libwrap' \
'--with-readline' \
'--with-ssl' \
'--without-docs' \
'--with-extra-charsets=all' \
'--with-plugins=max' \
'--with-embedded-server' \
make install

New package to install Calice Soft automatically

A CMake package is available here : This package enables to build the Calice Software in few steps. CMake handles the svn checkout/update and building.

Do (for ilcsoft v17-09 with g++4.8):

git clone
cd CaliceInstall
mkdir build
cd build
make -jX (X number of cores to use)

if you don t have a compiler compatible c++11 do -DENABLE_C++11=OFF.

If you have a different ilcsoft version, you need to modify the userXXX.cmake to the correct environment (typically copied from ILCSoft.cmake in ilcsoft installation folder). Once it is build all the includes, binaries and librairies are available in build/myInstall.

Manual Compilation

Every folders need to be compiled. The following will show an example (all other folders should be done similarly).

For ilcsoft with 17-09 with gcc 4.4 you can use this file: user-pro-test_x86_64_gcc44_sl6_v17-09.cmake

For ilcsoft with 17-09 with gcc 4.8 you can use this file: user-pro-test_x86_64_gcc48_sl6_v17-09.cmake

For ilcsoft with 17-10 with gcc 4.8 you can use this file: user-pro-test_x86_64_gcc48_sl6_v17-10.cmake

You need to modify the following line with the path of the folder where you have the calice software

     CACHE PATH "Path to CALICE Software" FORCE )

Create a folder build_xxx where the source files will be compiled (xxx represent the name of the source folder e.g build_calice_userlib). Then go inside the folder created and type:

$ mkdir build_xxx
$ cd build_xxx
$ cmake -C ../<cmakefile> ../xxx/

with xxx the name of the source folder (like ../calice_userlib/). It should display the following or similar, no errors you should be present at the end:

$ cmake -C ../user-pro-test_x86_64_gcc48_sl6_v17-10.cmake ../calice_sim/
loading initial cache file ../user-pro-test_x86_64_gcc48_sl6_v17-10.cmake
-- The C compiler identification is GNU 4.8.1
-- The CXX compiler identification is GNU 4.8.1
-- Check for working C compiler: /cvmfs/
-- Check for working C compiler: /cvmfs/ -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /cvmfs/
-- Check for working CXX compiler: /cvmfs/ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for ILCUTIL (1.3.0)
-- Found ILCUTIL: /afs/  
-- FLAGS -Wall;-Wextra;-Wshadow;-Weffc++;-pedantic;-Wno-long-long;-Wuninitialized;-Wl,-no-undefined
-- Performing Test CXX_FLAG_WORKS_-Wall
-- Performing Test CXX_FLAG_WORKS_-Wall - Success
-- Adding -Wall to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-Wextra
-- Performing Test CXX_FLAG_WORKS_-Wextra - Success
-- Adding -Wextra to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-Wshadow
-- Performing Test CXX_FLAG_WORKS_-Wshadow - Success
-- Adding -Wshadow to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-Weffc++
-- Performing Test CXX_FLAG_WORKS_-Weffc++ - Success
-- Adding -Weffc++ to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-pedantic
-- Performing Test CXX_FLAG_WORKS_-pedantic - Success
-- Adding -pedantic to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-Wno-long-long
-- Performing Test CXX_FLAG_WORKS_-Wno-long-long - Success
-- Adding -Wno-long-long to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-Wuninitialized
-- Performing Test CXX_FLAG_WORKS_-Wuninitialized - Success
-- Adding -Wuninitialized to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-Wl,-no-undefined
-- Performing Test CXX_FLAG_WORKS_-Wl,-no-undefined - Success
-- Adding -Wl,-no-undefined to CXX_FLAGS
-- Performing Test CXX_FLAG_WORKS_-std=c++11
-- Performing Test CXX_FLAG_WORKS_-std=c++11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Check for CLHEP (
-- Check for CLHEP_CLHEP_LIBRARY: /afs/ -- ok
-- Found CLHEP: /afs/  
-- Check for Marlin (1.9.0)
-- Check for Marlin_LIBRARIES: Marlin
-- Check for Marlin_MARLIN_LIBRARY: /afs/ -- ok
-- Found Marlin: /afs/  
-- Check for LCIO_LIBRARIES: lcio;sio
-- Check for LCIO_LCIO_LIBRARY: /afs/ -- ok
-- Check for LCIO_SIO_LIBRARY: /afs/ -- ok
-- Found LCIO: /afs/  
-- Check for GEAR (1.6.0)
-- Check for GEAR_LIBRARIES: gearsurf;gear;gearxml
-- Check for GEAR_GEARSURF_LIBRARY: /afs/ -- ok
-- Check for GEAR_GEAR_LIBRARY: /afs/ -- ok
-- Check for GEAR_GEARXML_LIBRARY: /afs/ -- ok
-- Found GEAR: /afs/  
-- Check for LCCD (1.3.1)
-- Check for LCCD_LIBRARIES: lccd
-- Check for LCCD_LCCD_LIBRARY: /afs/ -- ok
-- Found LCCD: /afs/  
-- Check for CondDBMySQL (0.9.6)
-- Check for CondDBMySQL_LIBRARIES: conddb
-- Check for CondDBMySQL_CONDDB_LIBRARY: /afs/ -- ok
-- Found CondDBMySQL: /afs/  
-- Found streamlog: /afs/  
-- Check for ROOT_CONFIG_EXECUTABLE: /afs/
-- Check for ROOT (6.06.02)
-- Check for ROOT_EXECUTABLE: /afs/
-- Check for ROOT_CINT_EXECUTABLE: /afs/
-- Check for ROOT_LIBRARIES: Core;RIO;Net;Hist;Graf;Graf3d;Gpad;Tree;Rint;Postscript;Matrix;Physics;MathCore;Thread;MultiProc
-- Check for ROOT_CORE_LIBRARY: /afs/ -- ok
-- Check for ROOT_RIO_LIBRARY: /afs/ -- ok
-- Check for ROOT_NET_LIBRARY: /afs/ -- ok
-- Check for ROOT_HIST_LIBRARY: /afs/ -- ok
-- Check for ROOT_GRAF_LIBRARY: /afs/ -- ok
-- Check for ROOT_GRAF3D_LIBRARY: /afs/ -- ok
-- Check for ROOT_GPAD_LIBRARY: /afs/ -- ok
-- Check for ROOT_TREE_LIBRARY: /afs/ -- ok
-- Check for ROOT_RINT_LIBRARY: /afs/ -- ok
-- Check for ROOT_POSTSCRIPT_LIBRARY: /afs/ -- ok
-- Check for ROOT_MATRIX_LIBRARY: /afs/ -- ok
-- Check for ROOT_PHYSICS_LIBRARY: /afs/ -- ok
-- Check for ROOT_MATHCORE_LIBRARY: /afs/ -- ok
-- Check for ROOT_THREAD_LIBRARY: /afs/ -- ok
-- Check for ROOT_MULTIPROC_LIBRARY: /afs/ -- ok
-- Check for /usr/lib64/
-- Found ROOT: /afs/  
-- /afs/
-- /afs/
-- Found Doxygen: /usr/bin/doxygen (found version "1.6.1") 
CMake Warning (dev) at /afs/ (SET):
  Cannot set "global_doc_exists": current scope has no parent.
Call Stack (most recent call first):
  CMakeLists.txt:170 (INCLUDE)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- ---------------------------------------
--  Documentation for CALICE_SIM     
--  CALICE_SIM_DOC_WORK_DIR      = /afs/
-- ---------------------------------------
-- -------------------------------------------------------------------------------
-- BUILD_SHARED_LIBS                       = ON
-- CMAKE_INSTALL_PREFIX                    = /afs/
-- CMAKE_BUILD_TYPE                        = RelWithDebInfo
-- BUILD_WITH_USE_LCCD                     = ON 
-- BUILD_WITH_USE_LCCD                     = ON 
-- LCIO_HOME                               =  
-- LCCD_HOME                               =  
-- Marlin_HOME                             =  
-- GEAR_HOME                               =  
-- CLHEP_HOME                              =  
-- ROOT_HOME                               =  
-- Change a value with: cmake -D<Variable>=<Value>
-- -------------------------------------------------------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /afs/

If no error occurs, then type:

$ make -j4 install

It will compile the source files and create a library to run with the software Marlin. The library is located in myInstall\lib\*.so.

All folders (RootTreeWriter, labview_converter, calice_userlib, calice_reco, calice_analysis) must be compiled!!! The order should be calice_userlib, labview_converter, calice_reco, calice_analysis, calice_sim, RootTreeWriter

Executing the reconstruction with Marlin

Marlin is the framework used for reconstruction. The advantage is that you can create any Processors that do anything you want on your data and keep the LCIO format. You can execute theses processors in a chain with a steering file (xml). The Main files are located at the bottom of the page.

More details on how to use Marlin here

Conversion from raw text file to LCIO (Labview Converter)

This will convert the text files we have from the testbeam to the LCIO format (Creates Collections that are used later for Marlin in the reconstruction). Later on, the goal is to avoid this step by converting directly during testbeam the data stream to LCIO.

In the xml file (CONV_Labview.xml), you need to change :

$ <parameter name="LCIOOutputFile" type="string">
$ /yourworkingdir/xxx.slcio
$ </parameter>

In <processor name="MyECALRootTreeGenerator" type="RootTreeGenerator3"> and <processor name="MyHCALRootTreeGenerator" type="RootTreeGenerator3">

$ <parameter name="OutputRootFileName" type="string">
$ rawdata_output_XCAL.root
$ </parameter>

In <processor name="MyLabviewConverter2" type="LabviewConverter2">

$ <parameter name="Data" type="string">
$ inputfile.txt
$ </parameter>

Once everything is filled, then you need to modify myMarlin to point to your lib directory:

$ MY_INSTALL=/yourinstall/

Then you can run the conversion by typing:

$ ./myMarlin CONV_Labview.xml

You will then obtain rootfiles (with raw data) and LCIO files with EBU and HBU Collections containing rawdata. You can see the structure of the LCIO file by typing :  $ dumpevent xxx.slcio #EventNr | less .


Once the conversion is done, the reconstruction can be made. It will create rootfiles and LCIO files containing physics observables (MIP, position, time...) that you can work on later doing analysis.

The file myMarlin should be modified to point to your libraries as above. The steering file Reco.xml need to be modified to change input and output paths for the LCIO files and rootfiles.

First change the input file (it should be the LCIO file that you obtain after the conversion):

$ <parameter name="LCIOInputFiles">
$ /inputfolder/FILENAME.slcio
$ </parameter>

The processor <processor name="GeoConditions" type="ConditionsProcessor"> contains all the paths related to the database ( It should not be modified. It will create a Collection for each database folder like Module description, module connection, gain, pedestal...

Then what needs to be modified is <processor name="Ahc2CalibrateProcessorXCAL" type="Ahc2CalibrateProcessor">. Few parameters need to be changed depending on what you want to do with the data.

<parameter name="MipCut" type="float"> 0.5 </parameter> is the MIP Threshold cut (has been disable into the source code).
<parameter name="PedestalSubtraction" type="bool">true </parameter> to substract Pedestal
<parameter name="ZeroSuppression" type="bool"> false </parameter> apply or not MIP cut
<parameter name="doErrorCalculation" type="bool">false </parameter> (not implemented yet)
<parameter name="doGainTemperatureCorrection" type="bool">false </parameter> (work in progress)
<parameter name="doMipTemperatureCorrection" type="bool">false </parameter> (work in progress)
<parameter name="doSaturationCorrection" type="bool">false </parameter> (not implemented, needed?? for this TB)
<parameter name="filterDeadCells" type="bool">false </parameter> (done)
<parameter name="filterDefaultCells" type="bool">false </parameter> (done)
<parameter name="doMipConversion" type="bool"> true </parameter> to convert raw ADC to MIP

Then for the output : <processor name="XcalRootTreeWriter" type="RootTreeWriter"> to get rootfiles and <processor name="MyLCIOOutputProcessor" type="LCIOOutputProcessor"> for LCIO output file. You need to modify the output path:

<parameter name="OutFileName" type="string">
<parameter name="LCIOOutputFile" type="string">

Then you can run the reconstruction by typing:

$ ./myMarlin Reco.xml

It will give you at the end rootfiles and LCIO files with physics observables.

Attention : The final rootfile can be "big" for root thus creating sometimes 2 TTree. The last cycle of the TTree should be read (example: bigtree;1 & bigtree;2 -> bigtree;2 should be the one read)

Event Display

An event display is available to look at the data. For that ilcsoft should be installed onto your local computer or afs and sourced. Then type:

$ glced &

A window CED event display should be opened. Then you can start it by running

$ ./myMarlin CED.xml

In CED.xml, you should provide the slcio file path you want to look at. Then by entering enter, you can loop through the events.


/!\ We are soon moving to the new simulation framework DD4hep. /!\

The page is available here :


If you need any kind of support, you can contact :


Engineering HCAL Prototype Software (last edited 2018-05-16 08:53:00 by huonglan)