User Tools

Site Tools


iridium_cluster:data

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
iridium_cluster:data [2014/06/04 13:16]
florido [GRID storage]
iridium_cluster:data [2016/06/28 15:34] (current)
florido [Generic storage]
Line 64: Line 64:
   rsync -avz -e 'ssh -l <​username>'​ --progress source destination   rsync -avz -e 'ssh -l <​username>'​ --progress source destination
 </​code>​ </​code>​
 +
 +However, the progress indicator is not very good and most of the time slows down 
 +the transfers in the purpose of writing to standard output. ​
 +Therefore I suggest you either **redirect the standard error and output**:
 +
 +<code bash>
 +  rsync -avz -e 'ssh -l <​username>'​ --progress source destination &> rsyncoutput.log
 +</​code>​
 +
 +Or even better, use **rsync own log file** instead:
 +<code bash>
 +  rsync -avz -e 'ssh -l <​username>'​ --log-file=rsyncoutput.log source destination
 +</​code>​
 +
 +check the contents of the logfile now and then to see the status:
 +<code bash>
 +  tail rsyncoutput.log
 +</​code>​
 +
  
  
Line 121: Line 140:
  
  
-=== Solution 5: dq2 tools ===+=== Solution 5: Rucio or dq2 tools ===
  
   * **Pros:**   * **Pros:**
     * works with GRID storage     * works with GRID storage
   * **Cons:**   * **Cons:**
-    * works with ATLAS datasets+    * works only with ATLAS datasets 
 + 
 +If you have and ATLAS dataset, the best is to transfer it to the local 
 +LUND Grid storage first, and then to the cluster directly if needed.  
 +To do that you need to submit a DaTRi request 
 + 
 +This page contains all you need to know on how to use the local storage:  
 +http://​www.hep.lu.se/​grid/​localgroupdisk.html 
 + 
 +To move the dataset from any ATLAS grid storage to Iridium, you are recommended to use Rucio, the successor of DQ2. Use the following:​ 
 + 
 +To enable RUCIO tools, you'll need to: 
 +  - copy and configure you GRID certificate on Iridium. 
 +  - run ''​setupATLAS''​ 
 +  - run ''​localSetupRucioClients''​ 
 +  - login to the GRID using ''​arcproxy -S atlas''​ or ''​voms-proxy-init''​ as one would do on //​lxplus.cern.ch//​.  
 + 
 +The RUCIO official documentation is here: http://​rucio.cern.ch/​cli_examples.html 
 + 
 +If you still want to use dq2 tools, here's how:
  
 To enable dq2 tools, you'll need to: To enable dq2 tools, you'll need to:
Line 134: Line 172:
   - login to the GRID using ''​arcproxy''​ or ''​voms-proxy-init''​ as one would do on //​lxplus.cern.ch//​ .    - login to the GRID using ''​arcproxy''​ or ''​voms-proxy-init''​ as one would do on //​lxplus.cern.ch//​ . 
  
-Information about dq2 on CERN Twiki:+Information about dq2 on CERN Twiki (only visible if you have a CERN account):
 https://​twiki.cern.ch/​twiki/​bin/​view/​AtlasComputing/​DQ2ClientsHowTo https://​twiki.cern.ch/​twiki/​bin/​view/​AtlasComputing/​DQ2ClientsHowTo
- 
  
 ---- ----
iridium_cluster/data.1401880569.txt.gz · Last modified: 2014/06/04 13:16 by florido