This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
aurora_cluster:storage [2017/04/12 10:36] florido [Additional dedicated storage] |
aurora_cluster:storage [2018/01/29 10:03] florido [Nuclear, Particle and Theorerical Physics] |
||
---|---|---|---|
Line 10: | Line 10: | ||
Please follow these rules to keep the storage tidy. | Please follow these rules to keep the storage tidy. | ||
- | - Whenever you need to move data to one of the shared folders below,\\ **create a folder with your username** and put the data in there.\\ In this way you keep your data away from unwanted people and you can set all the permissions you want on it. | + | - Whenever you need to move data to one of the shared folders below,\\ **create a folder with a name than can be understood by everyone** and put the data in there. |
- | - Example: <code:bash># create folder | + | - If the data has been generated by you or if it is for your exclusive use, **create a folder with your username** to accomodate the data. \\ In this way you keep your data away from unwanted people and you can set all the permissions you want on it. |
- | [pflorido@aurora1 hep]$ mkdir /projects/hep/nobackup/shared/pp/pflorido | + | |
- | # copy data | + | |
- | [pflorido@aurora1 hep]$ cp mydata /projects/hep/nobackup/shared/pp/pflorido/mydata | + | |
- | # list file permissions | + | |
- | [pflorido@aurora1 ~]$ ls -ltrah /projects/hep/nobackup/shared/pp/pflorido/mydata | + | |
- | -rw-r--r--. 1 pflorido hep 0 12 apr 12.32 /projects/hep/nobackup/shared/pp/pflorido/mydata | + | |
- | # change to read/write only user and group (no rights to everyone) | + | |
- | [pflorido@aurora1 ~]$ chmod 660 /projects/hep/nobackup/shared/pp/pflorido/mydata | + | |
- | # check applied permissions | + | |
- | [pflorido@aurora1 ~]$ ls -ltrah /projects/hep/nobackup/shared/pp/pflorido/mydata | + | |
- | -rw-rw----. 1 pflorido hep 0 12 apr 12.32 /projects/hep/nobackup/shared/pp/pflorido/mydata</code> | + | |
* Data found outside a folder named after a user it is a good candidate to be deleted. Of course there can be exceptions when needed. | * Data found outside a folder named after a user it is a good candidate to be deleted. Of course there can be exceptions when needed. | ||
- Remove data that you don't use, for example from the ''scratch'' folders. Be kind to your colleagues! storage has a cost. | - Remove data that you don't use, for example from the ''scratch'' folders. Be kind to your colleagues! storage has a cost. | ||
- | ===== Nuclear, Particle and Theorerical Physics ===== | + | - Follow the instructions at [[aurora_cluster:moving_data|Moving Data to and from the Aurora cluster]] when you start moving data around. |
+ | ===== Nuclear, Particle and Theoretical Physics ===== | ||
==== Dedicated Shared storage server on Aurora ==== | ==== Dedicated Shared storage server on Aurora ==== | ||
Line 32: | Line 22: | ||
/projects/hep/nobackup/ | /projects/hep/nobackup/ | ||
+ | | ||
+ | :!: ''/project/hep/fs2'' is a symbolic link to ''/project/hep/nobackup'' | ||
| | ||
The organization of data in the above folder is as follows, and users are encouraged to enforce it: | The organization of data in the above folder is as follows, and users are encouraged to enforce it: | ||
Line 45: | Line 37: | ||
^ ::: | ::: | ::: | ''/pp'' for Particle Physics users | | ^ ::: | ::: | ::: | ''/pp'' for Particle Physics users | | ||
^ ::: | ::: | ::: | ''/tp'' for Theoretical Physics users | | ^ ::: | ::: | ::: | ''/tp'' for Theoretical Physics users | | ||
- | ^ users | Copy of Iridium User homes | This folder contains a copy of each user's private home folder **on Iridium**. **DO NOT use this on Aurora**. This folder will be used as a backup until we plan some other Iridium backup. | /<username> each user her own folder| | ||
- | ^ ::: | ::: | ::: | ''/npguests/<username>'' for Nuclear Physics guests | | ||
- | ^ ::: | ::: | ::: | ''/ppguests/<username>'' for Particle Physics guests | | ||
Line 57: | Line 46: | ||
==== Dedicated storage for Particle Physics ==== | ==== Dedicated storage for Particle Physics ==== | ||
- | Description yet to come. | + | Particle Physics can share data to the [[:iridium_cluster|Iridium Cluster]] and has two additional storage servers listed below for their exclusive use. |
- | ==== Storage on Iridium ==== | + | Users not belonging to the division of Particle Physics might be warned to remove their data from those storages, unless otherwise agreed with the Particle Physics division. |
+ | |||
+ | === Storage on Iridium === | ||
The Iridium storage is still accessible at | The Iridium storage is still accessible at | ||
Line 67: | Line 58: | ||
Please refer to the documentation at [[iridium_cluster:basic_information]] for it. | Please refer to the documentation at [[iridium_cluster:basic_information]] for it. | ||
- | ==== Additional dedicated storage ==== | + | === Additional dedicated storage === |
There are two storage servers installed for the exclusive use of particle physics. | There are two storage servers installed for the exclusive use of particle physics. | ||
Line 75: | Line 66: | ||
^ Storage path ^ Intended usage ^ Size ^ | ^ Storage path ^ Intended usage ^ Size ^ | ||
- | | ''/project/hep/fs3/shared'' | data that is meant to stay for long time | 35TB | | + | | ''/projects/hep/fs3/shared'' | data that is meant to stay for long time | 35TB | |
- | | ''/project/hep/fs4/scratch'' | temporary data | 35TB | | + | | ''/projects/hep/fs4/scratch'' | temporary data | 35TB | |
+ | |||
+ | === Grid storage === | ||
+ | |||
+ | Lunarc hosts a grid storage for ATLAS. Currently we are migrating from a 20TB storage to a 100TB. | ||
+ | |||
+ | The local storage name is | ||
+ | |||
+ | |||
+ | One can interact through it using the Rucio client. | ||
+ | |||
+ | Details on how to access this storage are at this page: http://www.hep.lu.se/grid/localgroupdisk.html | ||
+ | |||
+ | More on how to do transfers at [[aurora_cluster:moving_data | Moving data to and from the cluster]] | ||
===== Mathematical Physics ===== | ===== Mathematical Physics ===== | ||
At the moment, only the user folders are available. We are negotiating a 10TB common storage with Lunarc. More information is soon to come. | At the moment, only the user folders are available. We are negotiating a 10TB common storage with Lunarc. More information is soon to come. |