This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
aurora_cluster:moving_data [2017/07/31 18:04] florido |
aurora_cluster:moving_data [2017/07/31 18:12] florido [Downloading/Uploading data to/from the GRID from Aurora] |
||
---|---|---|---|
Line 15: | Line 15: | ||
Small files, whose size are below 10GB, can be moved using aurora's frontend. | Small files, whose size are below 10GB, can be moved using aurora's frontend. | ||
- | Users of **Mathematical Physics**, as well as any other Lunarc user, can use their favorite tool to download and upload either from your own workstation, the Aurora front-end or the Aurora computing nodes. | + | Users of **Mathematical Physics**, as well as any other Lunarc user, can use their favorite tool to download and upload either from your own workstation or laptop to the Aurora front-end or the Aurora computing nodes. |
You can read about some of those tools on the [[iridium_cluster:data| Move data to and from the Iridium Cluster]] pages. | You can read about some of those tools on the [[iridium_cluster:data| Move data to and from the Iridium Cluster]] pages. | ||
Line 46: | Line 46: | ||
The picture below shows the various steps. {{ :aurora_cluster:datamovementflowchartbiggfx.png?600 |}} | The picture below shows the various steps. {{ :aurora_cluster:datamovementflowchartbiggfx.png?600 |}} | ||
- | ==== Uploading/Downloading data to/from an external source from Aurora ==== | + | ==== A. Uploading/Downloading data to/from an external source from Aurora ==== |
- Use your favourite download software. Some suggestions are available at [[iridium_cluster:data|Moving data to and from Iridium]]. If the tool is not available please ask Florido to install it on ''fs2-hep''. | - Use your favourite download software. Some suggestions are available at [[iridium_cluster:data|Moving data to and from Iridium]]. If the tool is not available please ask Florido to install it on ''fs2-hep''. | ||
- Use your home folder or one of the ''/projects/hep'' folders as a destination folder. Any other path is not writable by your user. The ''/tmp'' folder will be deleted regularly so you should not use that. ''/projects/hep/fs2'' is accessible by everyone, while ''/projects/hep/fs3'' and ''/projects/hep/fs4'' is dedicated storage for the ATLAS project. | - Use your home folder or one of the ''/projects/hep'' folders as a destination folder. Any other path is not writable by your user. The ''/tmp'' folder will be deleted regularly so you should not use that. ''/projects/hep/fs2'' is accessible by everyone, while ''/projects/hep/fs3'' and ''/projects/hep/fs4'' is dedicated storage for the ATLAS project. | ||
- | ==== Downloading/Uploading data to/from the GRID from Aurora ==== | + | ==== B. Downloading/Uploading data to/from the GRID from Aurora ==== |
For resources that can be stored on the GRID, you should definitely stage them on the Lund GRID storage instead, a few ways described at the link below, so that you can access them from all over the world in the fastest way possible. | For resources that can be stored on the GRID, you should definitely stage them on the Lund GRID storage instead, a few ways described at the link below, so that you can access them from all over the world in the fastest way possible. | ||
Please read the dedicated page [[aurora_cluster:moving_data:grid|Moving data between GRID and Aurora]] | Please read the dedicated page [[aurora_cluster:moving_data:grid|Moving data between GRID and Aurora]] |