aurora_cluster:storage
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
aurora_cluster:storage [2018/09/11 12:45] florido [Dedicated storage for Particle Physics] |
aurora_cluster:storage [2020/11/19 09:52] (current) florido [Dedicated division storage] |
||
|---|---|---|---|
| Line 6: | Line 6: | ||
| The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users. | The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users. | ||
| + | 2020-06: I am currently working on some kind of system to keep track of usage and do some minimalistic data management. More information will come. | ||
| ===== Basic rules ===== | ===== Basic rules ===== | ||
| Line 43: | Line 44: | ||
| ^ ''/cvmfs'' | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/geant4.cern.ch'' for Nuclear Physics users | | ^ ''/cvmfs'' | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/geant4.cern.ch'' for Nuclear Physics users | | ||
| ^ ::: | ::: | ::: | ::: | ''/atlas.cern.ch'' for Particle Physics users | | ^ ::: | ::: | ::: | ::: | ''/atlas.cern.ch'' for Particle Physics users | | ||
| - | |||
| - | ==== Dedicated storage for Particle Physics ==== | ||
| - | |||
| - | Particle Physics can share data to the [[:iridium_cluster|Iridium Cluster]] and has two additional storage servers listed below for their exclusive use. | ||
| - | |||
| - | Users not belonging to the division of Particle Physics might be warned to remove their data from those storages, unless otherwise agreed with the Particle Physics division. | ||
| === Storage on Iridium === | === Storage on Iridium === | ||
| Line 58: | Line 53: | ||
| Please refer to the documentation at [[iridium_cluster:basic_information]] for it. | Please refer to the documentation at [[iridium_cluster:basic_information]] for it. | ||
| - | === Additional dedicated storage === | + | ==== Dedicated division storage ==== |
| - | There are two storage servers installed for the **exclusive use of Particle Physics**. | + | Since 2016 we bought several dedicated storages. The ownership, use and permissions of which is described below. |
| - | The recommendation is to use these instead of ''/project/hep/nobackup'' so to give space to the other users of it. | + | |
| - | + | ||
| - | :!: ''/project/hep/fs2'' is a symbolic link to ''/project/hep/nobackup'' | + | |
| + | ^ Particle Physics ^^^^^^^ | ||
| ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ||
| | fs3 | 35TB | ''/projects/hep/fs3/shared'' | data that is meant to stay for long time | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | | | fs3 | 35TB | ''/projects/hep/fs3/shared'' | data that is meant to stay for long time | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | | ||
| Line 72: | Line 65: | ||
| | fs6 | 125TB | ''/projects/hep/fs6/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | | | fs6 | 125TB | ''/projects/hep/fs6/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | | ||
| | ::: | ::: | ''/projects/hep/fs6/scratch'' | temporary data | ::: | ::: | ::: | | | ::: | ::: | ''/projects/hep/fs6/scratch'' | temporary data | ::: | ::: | ::: | | ||
| - | | fs7 | 107TB | ''/projects/hep/fs7/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | LDMX | | | + | | fs7 | 107TB | ''/projects/hep/fs7/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | LDMX | Ask Ruth **before** using this space. | |
| - | | ::: | ::: | ''/projects/hep/fs7/scratch'' | temporary data | ::: | ::: | ::: | | + | | ::: | ::: | ''/projects/hep/fs7/scratch'' | temporary data | ::: | ::: | ::: | |
| + | | fs9 | 350TB | ''/projects/hep/fs9/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | | whole division | | ||
| + | | ::: | ::: | ''/projects/hep/fs9/scratch'' | temporary data | ::: | ::: | ::: | | ||
| + | | ::: | ::: | ''/projects/hep/fs9/users'' | data moved from ''/lunarc/nobackup/users'' | ::: | ::: | ::: | | ||
| + | | SNIC 2020/6-83 | 40TB | ''/lunarc/nobackup/projects/ldmx_simulations'' | Buffer for LDMX data | Slow disk suitable for low read/write frequency | LDMX | Time-limited Storage allocation on Lunarc CentreStorage. The data will be deleted at the end of the project or if the project is not renewed. | | ||
| - | === GRID storage === | + | ^ Nuclear Physics ^^^^^^^ |
| + | ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ||
| + | | fs8 | 350TB | ''/projects/hep/fs8/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | | whole division | | ||
| + | | ::: | ::: | ''/projects/hep/fs8/scratch'' | temporary data | ::: | ::: | ::: | | ||
| + | | ::: | ::: | ''/projects/hep/fs8/users'' | data moved from ''/lunarc/nobackup/users'' | ::: | ::: | ::: | | ||
| + | |||
| + | ^ Theoretical Particle Physics ^^^^^^^ | ||
| + | ^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ | ||
| + | | fs10 | 350TB | ''/projects/hep/fs10/shared'' | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | | whole division | | ||
| + | | ::: | ::: | ''/projects/hep/fs10/scratch'' | temporary data | ::: | ::: | ::: | | ||
| + | | ::: | ::: | ''/projects/hep/fs10/users'' | data moved from ''/lunarc/nobackup/users'' | ::: | ::: | ::: | | ||
| + | |||
| + | ==== GRID storage ==== | ||
| Lunarc hosts a GRID storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB. | Lunarc hosts a GRID storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB. | ||
| - | The local storage name is | + | The local storage GRID name is '' SE-SNIC-T2_LUND_LOCALGROUPDISK '' |
| - | + | ||
| | | ||
| - | One can interact through it using the Rucio client. | + | One can interact through it using the Rucio client. It cannot be mounted directly on the cluster. Data can only be moved to and fro. Access is restricted to **ATLAS** members. |
| Details on how to access this storage are at this page: http://www.hep.lu.se/grid/localgroupdisk.html | Details on how to access this storage are at this page: http://www.hep.lu.se/grid/localgroupdisk.html | ||
| Line 91: | Line 99: | ||
| ===== Mathematical Physics ===== | ===== Mathematical Physics ===== | ||
| - | At the moment, only the user folders are available. We are negotiating a 10TB common storage with Lunarc. More information is soon to come. | + | At the moment, only standard LUNARC user folders are available, to my knowledge. Please contact me if you want this information to be updated. |
aurora_cluster/storage.1536669934.txt.gz · Last modified: 2018/09/11 12:45 by florido
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
