User Tools

Site Tools


aurora_cluster:storage

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
aurora_cluster:storage [2017/04/13 09:15]
florido [Dedicated storage for Particle Physics]
aurora_cluster:storage [2020/06/15 14:43]
florido [Mathematical Physics]
Line 6: Line 6:
 The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users. The home folders are maintained and backed up by Lunarc, and they are in a special storage machine dedicated to users.
  
 +2020-06: I am currently working on some kind of system to keep track of usage and do some minimalistic data management. More information will come.
 ===== Basic rules ===== ===== Basic rules =====
  
Line 15: Line 16:
   - Remove data that you don't use, for example from the ''​scratch''​ folders. Be kind to your colleagues! storage has a cost.   - Remove data that you don't use, for example from the ''​scratch''​ folders. Be kind to your colleagues! storage has a cost.
   - Follow the instructions at [[aurora_cluster:​moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around.   - Follow the instructions at [[aurora_cluster:​moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around.
-===== Nuclear, Particle and Theorerical ​Physics =====+===== Nuclear, Particle and Theoretical ​Physics =====
  
 ==== Dedicated Shared storage server on Aurora ==== ==== Dedicated Shared storage server on Aurora ====
Line 37: Line 38:
 ^ ::: | ::: | ::: | ''/​pp''​ for Particle Physics users | ^ ::: | ::: | ::: | ''/​pp''​ for Particle Physics users |
 ^ ::: | ::: | ::: | ''/​tp''​ for Theoretical Physics users | ^ ::: | ::: | ::: | ''/​tp''​ for Theoretical Physics users |
-^ users | Copy of Iridium User homes | This folder contains a copy of each user's private home folder **on Iridium**. **DO NOT use this on Aurora**. This folder will be used as a backup until we plan some other Iridium backup. | /<​username>​ each user her own folder| 
-^ ::: | ::: | ::: | ''/​npguests/<​username>''​ for Nuclear Physics guests | 
-^ ::: | ::: | ::: | ''/​ppguests/<​username>''​ for Particle Physics guests | 
  
  
Line 46: Line 44:
 ^ ''/​cvmfs''​ | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/​geant4.cern.ch''​ for Nuclear Physics users | ^ ''/​cvmfs''​ | Special folder containing CERN maintained software | user cannot write | This special folder is dedicated to software provided by CERN. This folder is read-only. Usually the content of this folder are managed via specific scripts that a user can run. If you need to add some software that you cannot find, contact the administrators. | ''/​geant4.cern.ch''​ for Nuclear Physics users |
 ^ ::: | ::: | ::: | ::: | ''/​atlas.cern.ch''​ for Particle Physics users | ^ ::: | ::: | ::: | ::: | ''/​atlas.cern.ch''​ for Particle Physics users |
- 
-==== Dedicated storage for Particle Physics ==== 
- 
-Particle Physics can share data to the [[:​iridium_cluster|Iridium Cluster]] and has two additional storage servers listed below for their exclusive use. 
- 
-Users not belonging to the division of Particle Physics might be warned to remove their data from those storages, unless otherwise agreed with the Particle Physics division. 
  
 === Storage on Iridium === === Storage on Iridium ===
Line 61: Line 53:
 Please refer to the documentation at [[iridium_cluster:​basic_information]] for it. Please refer to the documentation at [[iridium_cluster:​basic_information]] for it.
  
-=== Additional dedicated ​storage ===+==== Dedicated division ​storage ​====
  
-There are two storage servers installed for the exclusive use of particle physics. +Since 2016 we bought several dedicated storages. The ownership, ​use and permissions ​of which is described below.
-The recommendation is to use these instead ​of ''/​project/​hep/​nobackup''​ so to give space to the other users of it.+
  
-:!: ''/​project/hep/fs2''​ is a symbolic link to ''/​project/​hep/​nobackup''​+^ Particle Physics ^^^^^^^ 
 +^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ 
 +| fs3 | 35TB  | ''/​projects/​hep/​fs3/​shared''​ | data that is meant to stay for long time | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| fs4 | 35TB  | ''/​projects/​hep/​fs4/​scratch''​ | temporary data | Very fast disk suitable for calculations | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| fs5 | 125TB | ''/​projects/​hep/​fs5/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| ::: | :::   | ''/​projects/hep/fs5/​scratch''​ | temporary data | ::: | ::: | ::: | 
 +| fs6 | 125TB | ''/​projects/​hep/​fs6/​shared'' ​| data that is meant to stay for long time | Slow disk suitable for low read/write frequency | ATLAS | Ask Caterina and Will **before** using this space. | 
 +| ::: | :::   ​| ​''/​projects/hep/​fs6/​scratch''​ | temporary data | ::: | ::: | ::: | 
 +| fs7 | 107TB | ''/​projects/​hep/​fs7/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency | LDMX | Ask Ruth **before** using this space. | 
 +| ::: | :::   | ''/​projects/​hep/​fs7/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| fs9 | 350TB | ''/​projects/​hep/​fs9/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency |  | whole division | 
 +| ::: | :::   | ''/​projects/​hep/​fs9/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| ::: | :::   | ''/​projects/​hep/​fs9/​users''​ | data moved from ''/​lunarc/nobackup/users'' ​| ::: | ::: | ::: | 
  
-^ Storage ​path ^ Intended usage ^ Size +^ Nuclear Physics ^^^^^^^ 
-| ''/​project/hep/fs3/​shared''​ | data that is meant to stay for long time | 35TB +^ Server ^ Size (Max) ^ Storage ​paths ^ Intended usage ^ Notes ^ Project ^ Remarks ​
-| ''/​project/hep/fs4/​scratch''​ | temporary data | 35TB |+| fs8 | 350TB | ''/​projects/hep/fs8/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency |  | whole division ​
 +| ::: | :::   | ''/​projects/hep/fs8/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| ::: | :::   | ''/​projects/​hep/​fs8/​users''​ | data moved from ''/​lunarc/​nobackup/​users''​ | ::: | ::: | ::: 
  
-=== Grid storage ===+^ Theoretical Particle Physics ^^^^^^^ 
 +^ Server ^ Size (Max) ^ Storage paths ^ Intended usage ^ Notes ^ Project ^ Remarks ^ 
 +| fs10 | 350TB | ''/​projects/​hep/​fs10/​shared''​ | data that is meant to stay for long time | Slow disk suitable for low read/write frequency |  | whole division | 
 +| ::: | :::   | ''/​projects/​hep/​fs10/​scratch''​ | temporary data | ::: | ::: | ::: |  
 +| ::: | :::   | ''/​projects/​hep/​fs10/​users''​ | data moved from ''/​lunarc/​nobackup/​users''​ | ::: | ::: | ::: | 
  
-Lunarc hosts a grid storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB.+==== GRID storage ==== 
 + 
 +Lunarc hosts a GRID storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB
 + 
 +The local storage GRID name is  ''​ SE-SNIC-T2_LUND_LOCALGROUPDISK ''​ 
 +   
 +One can interact through it using the Rucio client. It cannot be mounted directly on the cluster. Data can only be moved to and fro. Access is restricted to **ATLAS** members.
  
 Details on how to access this storage are at this page: http://​www.hep.lu.se/​grid/​localgroupdisk.html Details on how to access this storage are at this page: http://​www.hep.lu.se/​grid/​localgroupdisk.html
  
-More on how to do transfers at [[aurora_cluster:​moving_data]]+More on how to do transfers at [[aurora_cluster:​moving_data ​| Moving data to and from the cluster]] 
  
 ===== Mathematical Physics ===== ===== Mathematical Physics =====
  
-At the moment, only the user folders are available. ​We are negotiating a 10TB common storage with Lunarc. More information ​is soon to come.+At the moment, only standard LUNARC ​user folders are available, to my knowledgePlease contact me if you want this information to be updated.
aurora_cluster/storage.txt · Last modified: 2020/11/19 09:52 by florido

Accessibility Statement