How to use the BIRD (Batch Infrastucture Ressource at DESY)

<!> There is a bug in the setting.sh which leads to problems if you use a modified Z Shell color scheme. See here for help.

<!> If you encounter the following using grid commands

t2145220288:p28936: Fatal error: [Thread System] GLOBUSTHREAD: pthread_create() failed

[Thread System] insufficient memory (ENOMEM)zsh: 28936 abort (core dumped

Try to limit your stack size (zsh):

$ ulimit -s 100000

Anyhow submitting a job is very easy. You write a shell script which contains all the commands to be executed on the worker node. They are processed one after the other, like a batch (dt. Stapel), this is where the name comes from.

If you need more help have a look at the Batch Cluster For Dummies page.

Beside the commands there are a few switches, for example if you only want to run on 32-bit machines. You can either supply these switches as command line arguments, or you can include them inside your submitted shell script in the form of special comments that begin with the #$ character sequence.

# job-test.sh

#$ -l arch=amd64 
#$ -l os=sld5

echo "job test" > job-test.log

Submit this job with

qsub job-test.sh

and figure out where the job-test.log appears.

monitoring jobs

If you're not satisfied with the output of qmon or qstat, you might want to visit http://www.desy.de/~blutz/cluster/cluster.xml to see a list of the current job queues. The data is updated every 5 minutes.

excluding hosts

If you have the feeling that your jobs fail on a particular host, you can exclude it. (But don’t forget to inform the administrators, so that they can fix the problem for the sake of those who will come after you.)

-l h=\!fishyhost

See man 5 complex for more options. Note that this will only give you “complex – Sun Grid Engine complexes configuration file format” after sourcing the setup file /usr/sge/default/common/settings.sh. On a normal host, you may get “complex – basics of complex mathematics” instead.

Direct login (aka Interactive work)

Sometimes it's useful to run some commands interactively on the node similar to the one you plan to submit your jobs to. For that use the qlogin or qrsh commands of BIRD:

qlogin -q login.q

or, if you wish to specify a particular node

qlogin -q login.q@bird121.desy.de

dCache problems

If you have problems accessing your files see:

dCache

BatchCluster (last edited 2013-08-20 08:53:26 by Eduard Avetisyan)