pfwiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Running on the Aurora Cluster ======

In general, running on Aurora follows the rules stated on Lunarc manual pages.
All jobs must be ran under the //slurm// batch system.

There's only few important things one needs to know:

  * Our nodes are running in a special **partition** (a.k.a. queue) called ''hep''. Whenever in the documentation you're asked to specify a queue, use ''hep''
  * Our nodes run under a special **project** called ''HEP2016-1-1''. Whenever you're asked to insert a project name, that's the one.
  * Our storage is a bit different from the others and there are a few additional rules to use it. Please read [[aurora_cluster:storage]]

A typical direct submission to our nodes looks like:
<code bash>srun -p hep -A HEP2016-1-1 myscript.sh</code>

A typical slurm submission script ''slurmexample.sh'' written in bash will have this prologue:

<code bash>
#!/bin/bash
#
#SBATCH -A hep2016-1-1
#SBATCH -p hep 
#
hostname
srun echo $BASHPID;
</code>

to be run with the command:
<code bash>sbatch slurmexample.sh</code>

Basic usage of the batch system is described here:
  * Using the SLURM batch system: http://lunarc-documentation.readthedocs.io/en/latest/batch_system/
  * Batch system rules. Note: these rules might be slightly different for us since we have our own partition. http://lunarc-documentation.readthedocs.io/en/latest/batch_system_rules/


===== Interactive access to nodes for code testing =====

As said before, **it is not possible** to run your test code on the frontend on Aurora. But since everybody likes to test their code before submitting it to the batch system, //slurm// provides a nice way of using
a node allocation as an interactive session, just like we were doing on ''pptest-iridium'' and ''nptest-iridium''.

The interactive session is activated using the ''interactive'' command and a few options:

<code bash>
interactive -t 60 -p hep -A HEP2016-1-1
</code>

where ''-t 60'' is the time in minutes you want the interactive session to last. You can put as much as you want in the timer. Mind that whatever you're running will be killed after the specified time.

//slurm// will select a free node for you and open a bash terminal. From that moment on you can pretty much do the same as you were doing on Iridium testing nodes.

The interactive session is terminated when you issue the ''exit'' command.


===== Particle Physics users: CVMFS =====

Particle Physics users might want to use CVMFS to configure their own research software.
This system is now available on Aurora nodes and is the recommended way to do CERN-related analysis on the
cluster. This will make your code and init scripts the same on almost every cluster
who gives access to particle physicist in the whole SNIC and at any cluster at CERN, so learning how to do this on Aurora is a good experience.

**cvmfs** is a networked filesystem that hosts CERN software. The mount path is ''/cvmfs/''.
To initialize the environment, run the following lines of code:

<code bash>
export ATLAS_LOCAL_ROOT_BASE="/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase"
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
</code>

:!: At the moment there are some problems in enabling cvmfs scripts to be enabled at every login. If you add the above lines to to your ''.bash_profile'' file in your home folder, interactive access to the cluster will NOT work, and maybe even submission will be broken. I don't know the reason for this and it will be investigated. 
It is therefore **NOT RECOMMENDED** to add those lines in your ''.bash profile''. 
Run them **after** you login to an interactive session or add them in the prologue of your batch job.

===== Nuclear and Theoretical physics: loading libraries and tools =====

Aurora allows the user to load specific versions of compilers and tools using the excellent ''module'' system.
This system configures binary tools and library paths for you in a very flexible way.

Read more about it in Lunarc documentation here: http://lunarc-documentation.readthedocs.io/en/latest/aurora_modules/

If you are in need of a module that is not installed, please check this list:

https://github.com/hpcugent/easybuild-easyconfigs/tree/master/easybuild/easyconfigs

If the software you need **is in the list but NOT in Aurora**, report to Florido, and he will coordinate with Lunarc to provide such software. 

If that module **does not exist in the system nor in the above list**, you will have to **build it and configure yourself**. Read more about it in the [[#Custom software]] paragraph.

===== Custom software =====

Custom software can be installed in the ''/projects/hep/nobackup/software/<yourdivision>'' folder.
It is up to the user community to develop scripts to configure the environment.

Once the software is built and configurable we can consider creating our own modules to enable it. Ask Florido to help you in such development, and these modules will be shared on Aurora using the same mechanism as other software.
pfwiki

User Tools

Site Tools

Page Tools
