User Tools

Site Tools


aurora_cluster:storage

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
aurora_cluster:storage [2017/11/21 16:33]
florido
aurora_cluster:storage [2020/11/19 09:52] (current)
florido [Dedicated division storage]
Line 6: Line 6:
 The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users. The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users.
  
 +2020-06: I am currently working on some kind of system to keep track of usage and do some minimalistic data management. More information will come.
 ===== Basic rules ===== ===== Basic rules =====
  
Line 15: Line 16:
   - Remove data that you don't use, for example from the ''​scratch''​ folders. Be kind to your colleagues! storage has a cost.   - Remove data that you don't use, for example from the ''​scratch''​ folders. Be kind to your colleagues! storage has a cost.
   - Follow the instructions at [[aurora_cluster:​moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around.   - Follow the instructions at [[aurora_cluster:​moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around.
-===== Nuclear, Particle and Theorerical ​Physics =====+===== Nuclear, Particle and Theoretical ​Physics =====
  
 ==== Dedicated Shared storage server on Aurora ==== ==== Dedicated Shared storage server on Aurora ====
Line 43: Line 44:
 ^ ''/​cvmfs''​ | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/​geant4.cern.ch''​ for Nuclear Physics users | ^ ''/​cvmfs''​ | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/​geant4.cern.ch''​ for Nuclear Physics users |
 ^ ::: | ::: | ::: | ::: | ''/​atlas.cern.ch''​ for Particle Physics users | ^ ::: | ::: | ::: | ::: | ''/​atlas.cern.ch''​ for Particle Physics users |
- 
-==== Dedicated storage for Particle Physics ==== 
- 
-Particle Physics can share data to the [[:​iridium_cluster|Iridium Cluster]] and has two additional storage servers listed below for their exclusive use. 
- 
-Users not belonging to the division of Particle Physics might be warned to remove their data from those storages, unless otherwise agreed with the Particle Physics division. 
  
 === Storage on Iridium === === Storage on Iridium ===
Line 58: Line 53:
 Please refer to the documentation at [[iridium_cluster:​basic_information]] for it. Please refer to the documentation at [[iridium_cluster:​basic_information]] for it.
  
-=== Additional dedicated ​storage ===+==== Dedicated division ​storage ​====
  
-There are two storage servers installed for the exclusive use of particle physics. +Since 2016 we bought several dedicated storages. The ownership, ​use and permissions ​of which is described below.
-The recommendation is to use these instead ​of ''/​project/​hep/​nobackup''​ so to give space to the other users of it.+
  
-:!: ''/​project/hep/fs2''​ is a symbolic link to ''/​project/​hep/​nobackup''​+^ Particle Physics ^^^^^^^ 
 +^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ 
 +| fs3 | 35TB  | ''/​projects/​hep/​fs3/​shared''​ | data that is meant to stay for long time | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| fs4 | 35TB  | ''/​projects/​hep/​fs4/​scratch''​ | temporary data | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| fs5 | 125TB | ''/​projects/​hep/​fs5/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| ::: | :::   | ''/​projects/hep/fs5/​scratch''​ | temporary data | ::: | ::: | ::: | 
 +| fs6 | 125TB | ''/​projects/​hep/​fs6/​shared'' ​| data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| ::: | :::   ​| ​''/​projects/hep/​fs6/​scratch''​ | temporary data | ::: | ::: | ::: | 
 +| fs7 | 107TB | ''/​projects/​hep/​fs7/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | LDMX | Ask Ruth **before** using this space. | 
 +| ::: | :::   | ''/​projects/​hep/​fs7/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| fs9 | 350TB | ''/​projects/​hep/​fs9/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency |  | whole division | 
 +| ::: | :::   | ''/​projects/​hep/​fs9/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| ::: | :::   | ''/​projects/​hep/​fs9/​users''​ | data moved from ''/​lunarc/nobackup/users'' ​| ::: | ::: | ::: |  
 +| SNIC 2020/6-83 | 40TB | ''/​lunarc/​nobackup/​projects/​ldmx_simulations''​ | Buffer for LDMX data | Slow disk suitable for low read/write frequency | LDMX | Time-limited Storage allocation on Lunarc CentreStorage. The data will be deleted at the end of the project or if the project is not renewed. |
  
-^ Storage ​path ^ Intended usage ^ Size +^ Nuclear Physics ^^^^^^^ 
-| ''/​projects/​hep/​fs3/​shared''​ | data that is meant to stay for long time | 35TB +^ Server ^ Size (Max) ^ Storage ​paths ^ Intended usage ^ Notes ^ Project ^ Remarks ​
-| ''/​projects/​hep/​fs4/​scratch''​ | temporary data | 35TB |+| fs8 | 350TB | ''/​projects/​hep/​fs8/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency |  | whole division ​
 +| ::: | :::   | ''/​projects/​hep/​fs8/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| ::: | :::   | ''/​projects/​hep/​fs8/​users''​ | data moved from ''/​lunarc/​nobackup/​users''​ | ::: | ::: | ::: 
  
-=== Grid storage ===+^ Theoretical Particle Physics ^^^^^^^ 
 +^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ 
 +| fs10 | 350TB | ''/​projects/​hep/​fs10/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency |  | whole division | 
 +| ::: | :::   | ''/​projects/​hep/​fs10/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| ::: | :::   | ''/​projects/​hep/​fs10/​users''​ | data moved from ''/​lunarc/​nobackup/​users''​ | ::: | ::: | ::: | 
  
-Lunarc hosts a grid storage ​for ATLAS. Currently we are migrating from a 20TB storage to a 100TB.+==== GRID storage ​====
  
-The local storage name is +Lunarc hosts a GRID storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB. 
-  + 
 +The local storage ​GRID name is  ​''​ SE-SNIC-T2_LUND_LOCALGROUPDISK ''​
   ​   ​
-One can interact through it using the Rucio client. ​+One can interact through it using the Rucio client. It cannot be mounted directly on the cluster. Data can only be moved to and fro. Access is restricted to **ATLAS** members.
  
 Details on how to access this storage are at this page: http://​www.hep.lu.se/​grid/​localgroupdisk.html Details on how to access this storage are at this page: http://​www.hep.lu.se/​grid/​localgroupdisk.html
Line 85: Line 99:
 ===== Mathematical Physics ===== ===== Mathematical Physics =====
  
-At the moment, only the user folders are available. ​We are negotiating a 10TB common storage with Lunarc. More information ​is soon to come.+At the moment, only standard LUNARC ​user folders are available, to my knowledgePlease contact me if you want this information to be updated.
aurora_cluster/storage.1511282036.txt.gz · Last modified: 2017/11/21 16:33 by florido

Accessibility Statement