Differences

This shows you the differences between two versions of the page.

--- iridium_cluster:howtos_users [2013/08/27 18:20]
florido [Moving data to the cluster]
+++ iridium_cluster:howtos_users [2013/08/27 19:30]
florido
@@ Line 137: / Line 137: @@
 ^ Environment | Script to run | Description |
 ^ ATLAS Experiment environment | ''setupATLAS'' | Will setup all the neeeded environment variables for ATLAS experiment, and present a selection of other environments that the user can setup. |
+===== Tips'n'Tricks =====
+Suggestions on how to make your life easier when using the cluster.
 ==== Tips to speedup logging in ====
@@ Line 185: / Line 189: @@
 Where X and Y is the nodes you're allowed to run.
-note that with the above you will be requested to add as many password as each connection. For the moment being there is no way of making it better. Copying ssh keys to the gateway is not (yet) possible.
+note that with the above you will be requested to input as many password as the number of machines in the connection. A way to ease this pain is to [[##speedup_login_by_using_ssh_keys|copy ssh keys to the nodes.]]
+Copying ssh keys to the gateway is not (yet) possible, hence you will always need two passwords: one for the ssh key and one for the gateway.
 === Nuclear Physics ===
@@ Line 193: / Line 198: @@
 == References: ==
   * http://sshmenu.sourceforge.net/articles/transparent-mulithop.html
+==== Speedup login by using ssh keys ====
+An alternative method of authenticating via ssh is by using ssh keys. It will ease the pain of writing many passwords. The only password you will need is to unlock your key.
+:!: ** PLEASE DO NOT USE PASSWORDLESS KEYS. IT IS A GREAT SECURITY RISK. ** :!:
+Read about them here:
+https://wiki.archlinux.org/index.php/SSH_Keys
+==== How not to loose all your job because you closed a ssh terminal ====
+Use **screen**. //GNU screen// is an amazing tool that opens a remote terminal that is independent on your ssh connection. If the connection drops or you accidentally close the ssh window, it will still run your jobs on the cluster.
+A quick and dirty tutorial can be read [[:it_tips#screen|here]], but there's plenty more on the internet.
 ----
@@ Line 199: / Line 221: @@
 Please read the section [[#Common files organization]] before going through this section.
+==== Rules of thumb ====
+:!: **Please read this carefully.** :!:
+When moving data to the shared folders, please follow these common sense rules:
+  * Create folders for everything you want to share.
+  * If the data has been produced by you, is nice to create a folder with your name and place everything in it.
+  * If the data belongs to some specific experiment, dataset or the like, create a folder name that is consistent with that and that is easy for everybody to understand what that is about.
+  * Don't overdo. Only copy data you/your colleagues need. This is a shared facility.
+  * Don't remove other user's files unless you advice them and they're ok with it. This is a shared facility.
+  * Don't expect contents of the ''scratch'' folder to be always there. We still have no policy for that but we will have meetings in which we decide about it.
+==== Data transfer solutions ====
 Here's some solutions to move data to the cluster. 1-3 are generic data transfer tools. 4-5 are GRID oriented data transfer tools (mostly for Particle Physicists)
-==== Solution 1: scp,sftp,lsftp ====
+These marked with 8-) are my favourite  --- //[[:Florido Paganelli]] 2013/08/27 20:20//
-  * *Pros:*
+=== Solution 1: scp,sftp,lsftp ===
+  * **Pros:**
     * easy
     * only needs terminal
     * available almost everywhere
-  * *Cons:*
+    * progress indicator
+  * **Cons:**
     * not reliable. If connection goes down one must restart the entire transfer.
-    * no progress bar, one usually cannot know how long it will take
+    * does **not work** with GRID storage
-    * does *not work* with grid storage
+    * slow
 //Example://
+Moving ''ubuntu-12.04.2-desktop-amd64.iso'' from my local machine to ''n12.iridium''
 <code>
-  scp
+  scp ubuntu-12.04.2-desktop-i386.iso n12.iridium:/nfs/shared/pp/
 </code>
-==== Solution 2: rsync ====
+=== Solution 2: rsync  ===
-  * *Pros:*
+-)
+  * **Pros:**
     * Reliable. If connection goes down will resume from where it stopped.
     * Minimizes amount of transferred data by compressing it
     * only needs terminal
     * available on most GNU/Linux platforms
-  * *Cons:*
+    * a bit faster
+  * **Cons:**
     * Awkward command line
     * bad logs
-    * no progress bar
+    * poor progress indicator on many files
     * available on windows but needs special installation
-    * does *not work* with grid storage
+    * does **not work** with GRID storage
+//Example://
+Moving ''ubuntu-12.04.2-desktop-amd64.iso'' from my local machine to ''n12.iridium''
+<code>
+  rsync -avz --progress ubuntu-12.04.2-desktop-amd64.iso n12.iridium:/nfs/software/pp/
+</code>
-==== Solution 3: FileZilla ====
+=== Solution 3: FileZilla ===
-  * *Pros:*
+  * **Pros:**
     * Reliable. Tries to resume if connection went down.
     * Visual interface
     * Available for both GNU/Linux and windows
-  * *Cons:*
+  * **Cons:**
     * Visual interface :D
     * good logs
     * progress bar ^_^
-    * does *not work* with grid storage
+    * does **not work** with GRID storage
-==== Solution 4: NorduGrid ARC tools (arccp, arcls, arcrm) ====
+More about it: https://filezilla-project.org/download.php?type=client
-  * *Pros:*
+=== Solution 4: NorduGrid ARC tools (arccp, arcls, arcrm) ===
-    * works with grid storage
-  * *Cons:*
+  * **Pros:**
-    * doesn't work with ATLAS datasets (yet ;) )
+    * works with GRID storage
+  * **Cons:**
+    * doesn't work with ATLAS datasets (yet ;-) )
     * uncommon command line interface
-==== Solution 5: dq2 tools ====
+//Example://
+<code>
-  * *Pros:*
+</code>
-    * works with grid storage
-  * *Cons:*
+=== Solution 5: dq2 tools ===
+  * **Pros:**
+    * works with GRID storage
+  * **Cons:**
     * works with ATLAS datasets
     * uncommon command line interface (but some are used to it)
+//Example://
+<code>
+</code>

pfwiki

User Tools

Site Tools

Differences

Page Tools