This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
aurora_cluster:storage [2017/04/12 13:47] florido [Basic rules] |
aurora_cluster:storage [2020/06/15 14:36] florido [Dedicated storage] |
||
---|---|---|---|
Line 15: | Line 15: | ||
- Remove data that you don't use, for example from the ''scratch'' folders. Be kind to your colleagues! storage has a cost. | - Remove data that you don't use, for example from the ''scratch'' folders. Be kind to your colleagues! storage has a cost. | ||
- Follow the instructions at [[aurora_cluster:moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around. | - Follow the instructions at [[aurora_cluster:moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around. | ||
- | ===== Nuclear, Particle and Theorerical Physics ===== | + | ===== Nuclear, Particle and Theoretical Physics ===== |
==== Dedicated Shared storage server on Aurora ==== | ==== Dedicated Shared storage server on Aurora ==== | ||
Line 37: | Line 37: | ||
^ ::: | ::: | ::: | ''/pp'' for Particle Physics users | | ^ ::: | ::: | ::: | ''/pp'' for Particle Physics users | | ||
^ ::: | ::: | ::: | ''/tp'' for Theoretical Physics users | | ^ ::: | ::: | ::: | ''/tp'' for Theoretical Physics users | | ||
- | ^ users | Copy of Iridium User homes | This folder contains a copy of each user's private home folder **on Iridium**. **DO NOT use this on Aurora**. This folder will be used as a backup until we plan some other Iridium backup. | /<username> each user her own folder| | ||
- | ^ ::: | ::: | ::: | ''/npguests/<username>'' for Nuclear Physics guests | | ||
- | ^ ::: | ::: | ::: | ''/ppguests/<username>'' for Particle Physics guests | | ||
Line 46: | Line 43: | ||
^ ''/cvmfs'' | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/geant4.cern.ch'' for Nuclear Physics users | | ^ ''/cvmfs'' | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/geant4.cern.ch'' for Nuclear Physics users | | ||
^ ::: | ::: | ::: | ::: | ''/atlas.cern.ch'' for Particle Physics users | | ^ ::: | ::: | ::: | ::: | ''/atlas.cern.ch'' for Particle Physics users | | ||
- | |||
- | ==== Dedicated storage for Particle Physics ==== | ||
- | |||
- | Particle Physics can share data to the [[:iridium_cluster|Iridium Cluster]] and has two additional storage servers listed below for their exclusive use. | ||
- | |||
- | Users not belonging to the division of Particle Physics might be warned to remove their data from those storages, unless otherwise agreed with the Particle Physics division. | ||
=== Storage on Iridium === | === Storage on Iridium === | ||
Line 61: | Line 52: | ||
Please refer to the documentation at [[iridium_cluster:basic_information]] for it. | Please refer to the documentation at [[iridium_cluster:basic_information]] for it. | ||
- | === Additional dedicated storage === | + | ==== Dedicated division storage ==== |
- | There are two storage servers installed for the exclusive use of particle physics. | + | Since 2016 we bought several dedicated storages. The ownership, use and permissions of which is described below. |
- | The recommendation is to use these instead of ''/project/hep/nobackup'' so to give space to the other users of it. | + | |
- | :!: ''/project/hep/fs2'' is a symbolic link to ''/project/hep/nobackup'' | + | ^ Particle Physics ^^^^^^^ |
+ | ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ||
+ | | fs3 | 35TB | ''/projects/hep/fs3/shared'' | data that is meant to stay for long time | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | | ||
+ | | fs4 | 35TB | ''/projects/hep/fs4/scratch'' | temporary data | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | | ||
+ | | fs5 | 125TB | ''/projects/hep/fs5/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | | ||
+ | | ::: | ::: | ''/projects/hep/fs5/scratch'' | temporary data | ::: | ::: | ::: | | ||
+ | | fs6 | 125TB | ''/projects/hep/fs6/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | | ||
+ | | ::: | ::: | ''/projects/hep/fs6/scratch'' | temporary data | ::: | ::: | ::: | | ||
+ | | fs7 | 107TB | ''/projects/hep/fs7/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | LDMX | Ask Ruth **before** using this space. | | ||
+ | | ::: | ::: | ''/projects/hep/fs7/scratch'' | temporary data | ::: | ::: | ::: | | ||
+ | | fs9 | 350TB | ''/projects/hep/fs9/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | | whole division | | ||
+ | | ::: | ::: | ''/projects/hep/fs9/scratch'' | temporary data | ::: | ::: | ::: | | ||
+ | | ::: | ::: | ''/projects/hep/fs9/users'' | data moved from ''/lunarc/nobackup/users'' | ::: | ::: | ::: | | ||
+ | |||
+ | ^ Nuclear Physics ^^^^^^^ | ||
+ | ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ||
+ | | fs8 | 350TB | ''/projects/hep/fs8/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | | whole division | | ||
+ | | ::: | ::: | ''/projects/hep/fs8/scratch'' | temporary data | ::: | ::: | ::: | | ||
+ | | ::: | ::: | ''/projects/hep/fs8/users'' | data moved from ''/lunarc/nobackup/users'' | ::: | ::: | ::: | | ||
+ | |||
+ | ^ Theoretical Particle Physics ^^^^^^^ | ||
+ | ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ||
+ | | fs10 | 350TB | ''/projects/hep/fs10/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | | whole division | | ||
+ | | ::: | ::: | ''/projects/hep/fs10/scratch'' | temporary data | ::: | ::: | ::: | | ||
+ | | ::: | ::: | ''/projects/hep/fs10/users'' | data moved from ''/lunarc/nobackup/users'' | ::: | ::: | ::: | | ||
+ | |||
+ | === GRID storage === | ||
+ | |||
+ | Lunarc hosts a GRID storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB. | ||
+ | |||
+ | The local storage name is | ||
+ | |||
+ | |||
+ | One can interact through it using the Rucio client. | ||
+ | |||
+ | Details on how to access this storage are at this page: http://www.hep.lu.se/grid/localgroupdisk.html | ||
- | ^ Storage path ^ Intended usage ^ Size ^ | + | More on how to do transfers at [[aurora_cluster:moving_data | Moving data to and from the cluster]] |
- | | ''/project/hep/fs3/shared'' | data that is meant to stay for long time | 35TB | | + | |
- | | ''/project/hep/fs4/scratch'' | temporary data | 35TB | | + | |