User Tools

Site Tools


iridium_cluster:batch

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
iridium_cluster:batch [2014/05/09 11:16]
florido
iridium_cluster:batch [2016/08/12 18:57] (current)
florido [Available special variables]
Line 6: Line 6:
 The batch interface is used to submit multiple jobs to the cluster and let the machine do the scheduling for you, using a fair queuing system. The batch interface is used to submit multiple jobs to the cluster and let the machine do the scheduling for you, using a fair queuing system.
  
-Iridium currently supports two kind of batch interfaces:+Iridium currently supports two kinds of batch interfaces: 
 + 
 +===== The SLURM batch system ===== 
 + 
 +Can be used for old-style batch job submission.  
 +  * Files and environment must be set up by the user 
 +  * All paths are relative to cluster storage. 
 + 
 +  * A **tutorial** by Balázs Kónya can be found here: http://​www.hep.lu.se/​courses/​MNXB01/​2015/​MNXB01-tutorial-4a.pdf 
 + 
 +  * SLURM official pages: http://​slurm.schedmd.com/​slurm.html 
 + 
 +==== Available special variables ==== 
 +^ Name ^ Description ^ default value ^ 
 +| ''​$TMPDIR''​ | The temporary directory on a specific node that can be used by your script. Points at slurm temporary folder on that node. The stuff contained in this dir can be seen on any node (and also the test nodes) inside the ''/​nodestmp/​nX''​ folder, for each node ''​X''​ | ''/​slurmtmp''​ | 
 +===== The NorduGrid ARC GRID interface =====
  
-1) The [[http://​slurm.schedmd.com/​slurm.html|SLURM]] batch system 
-Can be used for old-style batch job submission. Files and environment must be set up by the user. All paths are relative to cluster storage. 
-2) The [[http://​www.nordugrid.org/​arc/​|NorduGrid ARC]] GRID interface 
 Part of this software has been designed and developed here in Lund. Part of this software has been designed and developed here in Lund.
-Can be used if input/​output files reside on GRID or remote storage (i.e. paths are relative to some remote cluster or storage ) and each job needs different but well established environments (i.e. software packages, environment variables etc.) so that one does not need to manually download the files and script the environment setup. The ARC interface will download the files and set up the environment automatically. ​It requires a GRID certificate and subscription to Virtual Organization (VO).+Can be used if input/​output files reside on GRID or remote storage (i.e. paths are relative to some remote cluster or storage ) and each job needs different but well established environments (i.e. software packages, environment variables etc.) so that one does not need to manually download the files and script the environment setup. ​ 
 + 
 +The ARC interface will  
 +  * Download/​upload input and output files automatically. 
 +  * Set up the environment automatically. ​ 
 + 
 +Use of this interface ​requires a **GRID certificate** and subscription to the [[https://​voms.ndgf.org:​8443/​voms/​nordugrid.org|NorduGrid ​Virtual Organization]] (VO). 
 + 
 +  * Nordugrid ARC homepage: http://​www.nordugrid.org/​arc/​ 
 + 
 +===== Information =====
  
-Details on how to use these two methods have been topic of a course by [[http://​www.hep.lu.se/​staff/​smirnova/​|Oxana Smirnova]] and [[http://​www.hep.lu.se/​staff/​konya/​|Balázs Kónya]] from Particle Physics here in Lund. The course material can be found here:+Details on how to use these two methods have been topic of a course by [[http://​www.hep.lu.se/​staff/​smirnova/​|Oxana Smirnova]] and [[http://​www.hep.lu.se/​staff/​konya/​|Balázs Kónya]] from Particle Physics here in Lund. The course material can be found here: http://​www.hep.lu.se/​courses/​grid/​
  
-http://www.hep.lu.se/​courses/​grid/​+The course has been held during the spring term 2014It can be repeated upon requestContact Oxana for any information.
  
-The course ​has been held during the spring ​term. It can be repeated upon requestContact Oxana for any information.+Another ​course ​on programming for science ​held during the Fall term has been touching these topicsThe contents of the course ​can be found here: http://www.hep.lu.se/​courses/​MNXB01/​
iridium_cluster/batch.1399626982.txt.gz · Last modified: 2014/05/09 11:16 by florido