ILCDirac for ILC users

problems or questions

Please use the jira or ilcdirac-support@cern.ch for iLCDirac problems or questions.

Registration in ILCDirac

Once you are in the VO ILC, you need to send a mail to "ilcdirac-register(at)cern.ch" stating your name, institute, corresponding contact (supervisor is usually enough) and project you are working on for registration.

For the details, please have look at this link:

https://twiki.cern.ch/twiki/bin/view/CLIC/IlcdiracRegistration

ILCDirac CVMFS installation

/cvmfs/clicdp.cern.ch/DIRAC

source /cvmfs/clicdp.cern.ch/DIRAC/bashrc

If you have already registered in ILDirac system as ilc_user with your grid certificate. You can initialize ilc_user role with the grid certificate password.

dirac-proxy-init -g ilc_user

Now you can start the ILCDirac file catalog client to access the detail DFC information.

dirac-dms-filecatalog-cli

You may type "help" to list all the available commands and online documentation.

FC:/> help

submit ilc_user jobs with ILCDirac


Example for ddsim job control

%> ilcdirac-version
ILCDirac Version: v28r0p2
With DIRAC version: v6r19p7


%> dirac-ilc-show-software

for example you may found - ddsim available version

   - ddsim 
     * ILCSoft-01-17-09 
     * ILCSoft-01-19-04_gcc49
     * ILCSoft-01-19-05_gcc49

ddsim = DDSim()
ddsim.setVersion("ILCSoft-01-19-05_gcc49")
ddsim.setSteeringFile("/cvmfs/ilc.desy.de/sw/ILDConfig/v01-19-05/StandardConfig/production/DDSim/ddsim_steer.py")


%> python run_user_ddsim.py

%> dirac-wms-job-status 17629821

%> dirac-wms-job-status -g Agroup

%> dirac-wms-job-status -f list_JobIDs.txt 

%> dirac-wms-job-get-output 17629821

run_user_ddsim.py

   1 from DIRAC.Core.Base import Script
   2 Script.parseCommandLine()
   3 
   4 from ILCDIRAC.Interfaces.API.DiracILC import DiracILC
   5 dirac = DiracILC(True,"user_example.rep")
   6 
   7 from ILCDIRAC.Interfaces.API.NewInterface.UserJob import UserJob
   8 job = UserJob()
   9 job.setName("user_example")
  10 job.setJobGroup("Agroup")
  11 job.setCPUTime(86400)
  12 
  13 # Note: you have to use the above tool "dirac-ilc-show-software" to find out the right/available tag name for your study.
  14 job.setILDConfig("v01-19-05-p01")
  15 
  16 # Note: you could use InputSnadBox to ship your local script onto the GRID working node. 
  17 job.setInputSandbox(["run_user_ddsim.py"])
  18 # Note: if your run on debug model, and log file may be too big, which could not be written out.
  19 job.setOutputSandbox(["*.log","*.xml","*.py "])
  20 lcoutput = []
  21 
  22 from ILCDIRAC.Interfaces.API.NewInterface.Applications import DDSim
  23 
  24 ddsim = DDSim()
  25 
  26 # Note: you have to use the above tool "dirac-ilc-show-software" to find out the right/available tag name for your study.
  27 ddsim.setVersion("ILCSoft-01-19-05_gcc49")
  28 
  29 ddsim.setDetectorModel("ILD_l5_v02")
  30 ddsim.setInputFile("LFN:/ilc/prod/ilc/mc-dbd/generated/1000-B1b_ws/6f/E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001.stdhep")
  31 ddsim.setSteeringFile("ddsim_steer.py")
  32 ddsim.setNumberOfEvents(10)
  33 ddsim.setRandomSeed(349709)
  34 ddsim.setEnergy(1000)
  35 ddsim.setStartFrom(1)
  36 ddsim.setOutputFile("sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_SIM.slcio")
  37 
  38 lcoutput.append("sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_SIM.slcio")
  39 
  40 job.append(ddsim)
  41 res = job.append(ddsim)
  42 if not res['OK']:
  43         print res['Not ok appending ddsim to job']
  44         quit()
  45 
  46 # The output file will be written into your HOME directory folder "ddsim/output"
  47 # i.e. "/ilc/user/<y>/<yourHOME>/ddsim/output/" in DFC.
  48 # For DESY local user, you may access "/pnfs/desy.de/ilc/user/<y>/<yourHOME>/ddsim/output/" on NAF2.
  49 job.setOutputData(lcoutput,"ddsim/output","DESY-SRM")
  50 print lcoutput
  51 
  52 print job.submit(dirac)


Example for Marlin job control

run_user_Marlin.py

   1 #!/usr/bin/env python
   2 from DIRAC.Core.Base import Script
   3 Script.parseCommandLine()
   4 
   5 from ILCDIRAC.Interfaces.API.DiracILC import DiracILC
   6 dirac = DiracILC(True,"user_example.rep")
   7 
   8 from ILCDIRAC.Interfaces.API.NewInterface.UserJob import UserJob
   9 job = UserJob()
  10 job.setName("user_example")
  11 job.setJobGroup("Agroup")
  12 job.setCPUTime(86400)
  13 
  14 # Note: you have to use the above tool "dirac-ilc-show-software" to find out the right/available tag name for your study.
  15 job.setILDConfig("v01-19-05-p01")
  16 
  17 job.setInputSandbox(["run_user_Marlin.py"])
  18 # Note: if your run on debug model, and log file may be too big, which could not be written out.
  19 job.setOutputSandbox(["*.log","*.xml","*.steer "]) 
  20 lcoutput = []
  21 from ILCDIRAC.Interfaces.API.NewInterface.Applications import Marlin
  22 
  23 ma = Marlin()
  24 
  25 # Note: you have to use the above tool "dirac-ilc-show-software" to find out the right/available tag name for your study.
  26 ma.setVersion("ILCSoft-01-19-05_gcc49")
  27 
  28 ma.setInputFile("LFN:/ilc/user/s/shaojun/TEST_ilc_user/sim/sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_SIM.slcio")
  29 ma.setSteeringFile("MarlinStdReco.xml")
  30 ma.setExtraCLIArguments( "--constant.lcgeo_DIR=$lcgeo_DIR --constant.DetectorModel=ILD_l5_o1_v02 --constant.OutputBaseName=sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100 --global.MaxRecordNumber=0" )
  31 ma.setLogFile("marlin.log")
  32 ma.setEnergy(1000)
  33 ma.setOutputRecFile("sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_REC.slcio")
  34 ma.setOutputDstFile("sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_DST.slcio")
  35 
  36 lcoutput.append("sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_REC.slcio")
  37 lcoutput.append("sv01-19-05.mILD_l5_v02.LQGSP_BERT.E1000-B1b_ws.Pvvxyyx.Gwhizard-1.95.eL.pR.I35867.001_evt100_DST.slcio")
  38 
  39 res = job.append(ma)
  40 if not res['OK']:
  41      print res['Not ok appending Marlin to job']
  42      quit()
  43 
  44 
  45 job.setOutputData(lcoutput,"Marlin/rec","DESY-SRM")
  46 print lcoutput
  47 
  48 print job.submit(dirac)

You may access the ILDConfig HEAD version from this link: https://github.com/iLCSoft/ILDConfig


Example for submitting mass UserJobs


ILCDirac Monitoring Webinterface


ILCDirac for ilc_prod role

Akiya Miyamoto (KEK Japan)
Shaojun Lu (DESY Germany)

In case someone will involve this task in the future, please contact the ILD mass production responsible person for the knowledge of running production job on ILCDirac system as ilc_prod role. They have the knowledge and the scripts at hand. These information will not be published on the wiki page here, instead the persons and groups, that you may consult, will be published only.

ILCDirac (last edited 2019-04-12 08:59:37 by ShaojunLu)