User Tools

Site Tools



This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
iridium_cluster:data [2014/06/04 11:16]
florido [GRID storage]
iridium_cluster:data [2016/06/28 13:34] (current)
florido [Generic storage]
Line 64: Line 64:
   rsync -avz -e 'ssh -l <​username>'​ --progress source destination   rsync -avz -e 'ssh -l <​username>'​ --progress source destination
 </​code>​ </​code>​
 +However, the progress indicator is not very good and most of the time slows down 
 +the transfers in the purpose of writing to standard output. ​
 +Therefore I suggest you either **redirect the standard error and output**:
 +<code bash>
 +  rsync -avz -e 'ssh -l <​username>'​ --progress source destination &> rsyncoutput.log
 +Or even better, use **rsync own log file** instead:
 +<code bash>
 +  rsync -avz -e 'ssh -l <​username>'​ --log-file=rsyncoutput.log source destination
 +check the contents of the logfile now and then to see the status:
 +<code bash>
 +  tail rsyncoutput.log
Line 121: Line 140:
-=== Solution 5: dq2 tools ===+=== Solution 5: Rucio or dq2 tools ===
   * **Pros:**   * **Pros:**
     * works with GRID storage     * works with GRID storage
   * **Cons:**   * **Cons:**
-    * works with ATLAS datasets+    * works only with ATLAS datasets 
 +If you have and ATLAS dataset, the best is to transfer it to the local 
 +LUND Grid storage first, and then to the cluster directly if needed.  
 +To do that you need to submit a DaTRi request 
 +This page contains all you need to know on how to use the local storage:  
 +To move the dataset from any ATLAS grid storage to Iridium, you are recommended to use Rucio, the successor of DQ2. Use the following:​ 
 +To enable RUCIO tools, you'll need to: 
 +  - copy and configure you GRID certificate on Iridium. 
 +  - run ''​setupATLAS''​ 
 +  - run ''​localSetupRucioClients''​ 
 +  - login to the GRID using ''​arcproxy -S atlas''​ or ''​voms-proxy-init''​ as one would do on //​​.  
 +The RUCIO official documentation is here: http://​​cli_examples.html 
 +If you still want to use dq2 tools, here's how:
 To enable dq2 tools, you'll need to: To enable dq2 tools, you'll need to:
Line 134: Line 172:
   - login to the GRID using ''​arcproxy''​ or ''​voms-proxy-init''​ as one would do on //​​ .    - login to the GRID using ''​arcproxy''​ or ''​voms-proxy-init''​ as one would do on //​​ . 
-Information about dq2 on CERN Twiki:+Information about dq2 on CERN Twiki (only visible if you have a CERN account):
 https://​​twiki/​bin/​view/​AtlasComputing/​DQ2ClientsHowTo https://​​twiki/​bin/​view/​AtlasComputing/​DQ2ClientsHowTo
 ---- ----
iridium_cluster/data.1401880569.txt.gz · Last modified: 2014/06/04 11:16 by florido