User Tools

Site Tools


iridium_cluster:howtos_users

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
iridium_cluster:howtos_users [2013/08/27 18:35]
florido
iridium_cluster:howtos_users [2013/10/15 15:57] (current)
florido [Assigned Nodes]
Line 137: Line 137:
 ^ Environment | Script to run | Description | ^ Environment | Script to run | Description |
 ^ ATLAS Experiment environment | ''​setupATLAS''​ | Will setup all the neeeded environment variables for ATLAS experiment, and present a selection of other environments that the user can setup. | ^ ATLAS Experiment environment | ''​setupATLAS''​ | Will setup all the neeeded environment variables for ATLAS experiment, and present a selection of other environments that the user can setup. |
 +
 +===== Tips'​n'​Tricks =====
 +
 +Suggestions on how to make your life easier when using the cluster.
  
 ==== Tips to speedup logging in ==== ==== Tips to speedup logging in ====
Line 185: Line 189:
 Where X and Y is the nodes you're allowed to run. Where X and Y is the nodes you're allowed to run.
  
-note that with the above you will be requested to add as many password as each connection. ​For the moment being there is no way of making it better. Copying ssh keys to the gateway is not (yet) possible.+note that with the above you will be requested to input as many password as the number of machines in the connection. ​way to ease this pain is to [[##​speedup_login_by_using_ssh_keys|copy ssh keys to the nodes.]] 
 +Copying ssh keys to the gateway is not (yet) possible, hence you will always need two passwords: one for the ssh key and one for the gateway.
  
 === Nuclear Physics === === Nuclear Physics ===
Line 193: Line 198:
 == References: == == References: ==
   * http://​sshmenu.sourceforge.net/​articles/​transparent-mulithop.html   * http://​sshmenu.sourceforge.net/​articles/​transparent-mulithop.html
 +
 +==== Speedup login by using ssh keys ====
 + 
 +An alternative method of authenticating via ssh is by using ssh keys. It will ease the pain of writing many passwords. The only password you will need is to unlock your key.
 +
 +:!: ** PLEASE DO NOT USE PASSWORDLESS KEYS. IT IS A GREAT SECURITY RISK. ** :!:
 +
 +Read about them here:
 +
 +https://​wiki.archlinux.org/​index.php/​SSH_Keys
 +
 +
 +==== How not to loose all your job because you closed a ssh terminal ====
 +
 +Use **screen**. //GNU screen// is an amazing tool that opens a remote terminal that is independent on your ssh connection. If the connection drops or you accidentally close the ssh window, it will still run your jobs on the cluster.
 +
 +A quick and dirty tutorial can be read [[:​it_tips#​screen|here]],​ but there'​s plenty more on the internet.
  
 ---- ----
Line 199: Line 221:
  
 Please read the section [[#Common files organization]] before going through this section. Please read the section [[#Common files organization]] before going through this section.
 +
 +==== Rules of thumb ====
 +
 +:!: **Please read this carefully.** :!:
 +
 +When moving data to the shared folders, please follow these common sense rules:
 +  * Create folders for everything you want to share.
 +  * If the data has been produced by you, is nice to create a folder with your name and place everything in it.
 +  * If the data belongs to some specific experiment, dataset or the like, create a folder name that is consistent with that and that is easy for everybody to understand what that is about.
 +  * Don't overdo. Only copy data you/your colleagues need. This is a shared facility.
 +  * Don't remove other user's files unless you advice them and they'​re ok with it. This is a shared facility.
 +  * Don't expect contents of the ''​scratch''​ folder to be always there. We still have no policy for that but we will have meetings in which we decide about it.
 +
 +==== Data transfer solutions ====
  
 Here's some solutions to move data to the cluster. 1-3 are generic data transfer tools. 4-5 are GRID oriented data transfer tools (mostly for Particle Physicists) Here's some solutions to move data to the cluster. 1-3 are generic data transfer tools. 4-5 are GRID oriented data transfer tools (mostly for Particle Physicists)
Line 204: Line 240:
 These marked with 8-) are my favourite ​ --- //​[[:​Florido Paganelli]] 2013/08/27 20:20// These marked with 8-) are my favourite ​ --- //​[[:​Florido Paganelli]] 2013/08/27 20:20//
  
-==== Solution 1: scp,​sftp,​lsftp ​====+=== Solution 1: scp,​sftp,​lsftp ===
  
   * **Pros:​** ​   * **Pros:​** ​
Line 217: Line 253:
  
 //​Example://​ //​Example://​
 +
 +Moving ''​ubuntu-12.04.2-desktop-amd64.iso''​ from my local machine to ''​n12.iridium''​
 +
 <​code>​ <​code>​
   scp ubuntu-12.04.2-desktop-i386.iso n12.iridium:/​nfs/​shared/​pp/​   scp ubuntu-12.04.2-desktop-i386.iso n12.iridium:/​nfs/​shared/​pp/​
 </​code>​ </​code>​
  
-==== Solution 2: rsync  ​====+=== Solution 2: rsync  ===
  
 8-) 8-)
Line 239: Line 278:
  
 //​Example://​ //​Example://​
 +
 +Moving ''​ubuntu-12.04.2-desktop-amd64.iso''​ from my local machine to ''​n12.iridium''​
 +
 <​code>​ <​code>​
   rsync -avz --progress ubuntu-12.04.2-desktop-amd64.iso n12.iridium:/​nfs/​software/​pp/​   rsync -avz --progress ubuntu-12.04.2-desktop-amd64.iso n12.iridium:/​nfs/​software/​pp/​
Line 244: Line 286:
  
  
-==== Solution 3: FileZilla ​====+=== Solution 3: FileZilla ===
  
   * **Pros:**   * **Pros:**
Line 258: Line 300:
 More about it: https://​filezilla-project.org/​download.php?​type=client More about it: https://​filezilla-project.org/​download.php?​type=client
  
-==== Solution 4: NorduGrid ARC tools (arccp, arcls, arcrm) ​====+=== Solution 4: NorduGrid ARC tools (arccp, arcls, arcrm) ===
  
   * **Pros:**   * **Pros:**
     * works with GRID storage     * works with GRID storage
   * **Cons:**   * **Cons:**
-    * doesn'​t work with ATLAS datasets (yet ;) )+    * doesn'​t work with ATLAS datasets (yet ;-) )
     * uncommon command line interface     * uncommon command line interface
  
Line 272: Line 314:
  
  
-==== Solution 5: dq2 tools ====+=== Solution 5: dq2 tools ===
  
   * **Pros:**   * **Pros:**
Line 292: Line 334:
 ^Researcher^ nodes ^  ^Researcher^ nodes ^ 
 ^Pico | n1, [[n7]] | ^Pico | n1, [[n7]] |
-^Jingwei ​| n2, n8 |+^Lene | n2, n8 
 +^Oleksandr | n6, n12 |
 ^Anthony | n3, n9 | ^Anthony | n3, n9 |
 ^Anders | n4, n10 | ^Anders | n4, n10 |
iridium_cluster/howtos_users.1377628555.txt.gz · Last modified: 2013/08/27 18:35 by florido

Accessibility Statement