This is an old revision of the document!
Storage in the Aurora cluster is organized as detailed by Lunarc. Read the following: http://lunarc-documentation.readthedocs.io/en/latest/quick_reference/#file-systems
The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users.
Please follow these rules to keep the storage tidy.
scratch
folders. Be kind to your colleagues! storage has a cost.
In particular, the HEP nodes dedicated storage usable by members of all divisions is accessible on hep nodes au[193-216]
at this path:
/projects/hep/nobackup/
/project/hep/fs2
is a symbolic link to /project/hep/nobackup
The organization of data in the above folder is as follows, and users are encouraged to enforce it:
Folder name | Folder purpose | Description | Subfolders |
---|---|---|---|
software | Application software | This folder hosts software that is not accessible via cvmfs (see later). This usually includes user/project specific libraries and frameworks. | /np for Nuclear Physics users |
/pp for Particle Physics users |
|||
/tp for Theoretical Physics users |
|||
shared | Data that will stay for long term | This folder should be used for long-term stored data. For example, long-term data sets or data needed for the whole duration of a phd project or shared among people belonging to the same research group. | /np for Nuclear Physics users |
/pp for Particle Physics users |
|||
/tp for Theoretical Physics users |
|||
scratch | Data that will stay for short term | This folder should be used for short-term stored data. For example, data needed for a week long calculation or temporary calculation. This folder should be considered unreliable as its contents will be purged from time to time. The cleanup interval is yet to be decided | /np for Nuclear Physics users |
/pp for Particle Physics users |
|||
/tp for Theoretical Physics users |
Other special folders:
/cvmfs | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | /geant4.cern.ch for Nuclear Physics users |
---|---|---|---|---|
/atlas.cern.ch for Particle Physics users |
The Iridium storage is still accessible at
/projects/hep/iridium/
Please refer to the documentation at basic_information for it.
Since 2016 we bought several dedicated storages. The ownership, use and permissions of which is described below.
Particle Physics | ||||||
---|---|---|---|---|---|---|
Server | Size (Max) | Storage paths | Intended usage | Notes | Project | Remarks |
fs3 | 35TB | /projects/hep/fs3/shared | data that is meant to stay for long time | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will before using this space. |
fs4 | 35TB | /projects/hep/fs4/scratch | temporary data | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will before using this space. |
fs5 | 125TB | /projects/hep/fs5/shared | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will before using this space. |
/projects/hep/fs5/scratch | temporary data | |||||
fs6 | 125TB | /projects/hep/fs6/shared | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will before using this space. |
/projects/hep/fs6/scratch | temporary data | |||||
fs7 | 107TB | /projects/hep/fs7/shared | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | LDMX | Ask Ruth before using this space. |
/projects/hep/fs7/scratch | temporary data | |||||
fs9 | 350TB | /projects/hep/fs9/shared | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | whole division | |
/projects/hep/fs9/scratch | temporary data | |||||
/projects/hep/fs9/users | data moved from /lunarc/nobackup/users |
Nuclear Physics | ||||||
---|---|---|---|---|---|---|
Server | Size (Max) | Storage paths | Intended usage | Notes | Project | Remarks |
fs8 | 350TB | /projects/hep/fs8/shared | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | whole division | |
/projects/hep/fs8/scratch | temporary data | |||||
/projects/hep/fs8/users | data moved from /lunarc/nobackup/users |
Theoretical Particle Physics | ||||||
---|---|---|---|---|---|---|
Server | Size (Max) | Storage paths | Intended usage | Notes | Project | Remarks |
fs10 | 350TB | /projects/hep/fs10/shared | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | whole division | |
/projects/hep/fs10/scratch | temporary data | |||||
/projects/hep/fs10/users | data moved from /lunarc/nobackup/users |
Lunarc hosts a GRID storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB.
The local storage name is
One can interact through it using the Rucio client.
Details on how to access this storage are at this page: http://www.hep.lu.se/grid/localgroupdisk.html
More on how to do transfers at Moving data to and from the cluster
At the moment, only the user folders are available. We are negotiating a 10TB common storage with Lunarc. More information is soon to come.