pfwiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Iridium cluster Work In Progress ======

Iridium cluster is a computing facility to serve researchers
from Particle Physics and Nuclear Physics at Lund University.

It is currently maintained by [[:about_Florido_Paganelli|Florido Paganelli]] and [[http://nuclearphysics.nuclear.lu.se/Staff/StaffSearch.asp?id=208&logo=no|Luis Sarmiento (aka Pico)]] with the help of the
[[http://www.lunarc.lu.se|Lunarc]] team.

===== Roadmap =====

:!: new schedule set September 9th, 2015 :!:

  * 1) Reconfigure service disks on frontend with a more flexible technology, e.g. lvm. 
    * Due w38
  * 2) Reinstall service machines with lxc instead of kvm/virtualization, 
    * Due w38
  * 3) Connect to the newly bought storage at Lunarc 
    * Due October 2015
  * 4) Connect to the default storage DDN at Lunarc 
    * Due October 2015
  * 5) Configure the new nodes to integrate into Lunarc 
    * Due January 2016
  * 6) Configure access to be integrated into Lunarc 
    * Due January 2016




==== DONE (as 9th Sep 2015) ====
  * 1) understand how to interact with existing grid storage present at Lunarc. Update: Lunarc suggested to benchmark current connection. If not enough, interact with Jens to understand how direct connection can be achieved. Lunarc only has data, no metadata or index.
    * Outcome: users prefer to interact directly with the data and move it on the cluster storage. Use of FAX is not possible yet both because it's an experimental technology and also because FAX is only installed in Tier0 or Tier1. The storage should be one of these.
  * 4) Start batch/grid jobs tests with users from particle physics and nuclear physics. In this phase we will be able to see how resource management should be done for an optimal use of the cluster. Missing: ATLAS RTEs.
    * Outcome: not so succesful due to the cluster not being a tier2 and its connection to grid storage not optimal. Also, disk problems prevented optimal operation.
  * 5) Setup n1 and n2 as test nodes. This includes:
    * Configure direct connection of nodes to the internet DONE
    * LDAP+OTP authentication on nodes. Status: sent domain name to lunarc. Received first bit of info from Lunarc. NOT DONE: impractical for use, there was no time to set it up. To be scheduled for next iteration and new Iridium nodes.



==== DONE (as 17th Mar 2014) ====

  * 3) install grid interfaces for researchers to run test grid jobs.

==== DONE (as 31st Jan 2014) ====
  * 2) install batch system.
    * add to salt config:
      * create folders for slurm
      * add slurm ports to iptables, wrt machines/services: 6817,6818,7321
    * set correct node values in ''slurm.conf''
  

==== DONE (as 29th Jan 2014) ====

  * Physical direct connection of nodes n1/n2 to the internet done by Robert
  * started batch system installation
    * created user for slurm
    * used n10 as guinea pig for configuration

==== DONE (as 15th Oct 2013) ====

  * 1) Understand LDAP authentication as it is done in Lunarc

==== DONE (as 15th Sept 2013) ====

  * 1) Understand LDAP authentication as it is done in Lunarc
  * start testing programme. Each early tester gets one or two nodes.

==== DONE (as 20th August 2013) ====
  * 1) setup a direct ssh access to at least one of the computing nodes
  * 2) configure storage server with minimal services for users (e.g. home folders)
  * 3) install application software for researchers to run test jobs. This will include the use of CERNVM and needs some coordination with Lunarc. 
    * independent from Lunarc. Meeting suggested to find alternative solutions. Luis set up of SALT.
    * Luis prepared automation of installation on all nodes.
    * cvmfs installation on one node succesful. installation automation is ongoing task.
===== Tech documents =====

Description of the cluster. Some of these documents might have restricted access. Contact me if
you need to access those.

  * Iridium cluster summary [to be uploaded]
  * graphical view of cluster elements {{::cluster_plan.png?linkonly}} **obsolete**
  * graphical view of planned cluster software {{:cluster_plan-sw.png?linkonly}} **obsolete**
===== Captain's log =====

All the work that has been done, day by day.

Moved to another page because it was to big: [[iridium_cluster:captainslog]]

===== Issues =====

  * Faulty PSU on node Chassis 1 replaced with Alarik spare. Waiting for replacement part to come.
  * Faulty motherboard on n11 was completely replaced.

----

===== Useful links =====

==== Hardware related ====
  * Information on the RAID controller on storage: 
    * http://hwraid.le-vert.net/wiki/LSIMegaRAIDSAS
    * Utility mentioned up there can be found on LSI logic website painfully. I finally find it here:
      * http://www.lsi.com/support/Pages/download-results.aspx?keyword=megacli
    * Cheat sheet for the above utility:
      * http://erikimh.com/megacli-cheatsheet/

==== System related ====

  * linux network bridge setup: http://wiki.libvirt.org/page/Networking#Host_configuration
  * CentOS basic firewall config: http://wiki.centos.org/HowTos/Network/IPTables
  * CentOS iptables logging: http://www.tokiwinter.com/adding-logging-to-iptables-under-centos/
  * CentOS NAT configuration: http://www.centos.org/docs/4/html/rhel-sg-en-4/s1-firewall-ipt-fwd.html
  * CentOS NIS configuration: http://www.server-world.info/en/note?os=CentOS_6&p=nis&f=1
  * CentOS NIS + IPTABLES: http://sysadmin-notepad.blogspot.se/2013/06/nis-server-setup-on-rhelcentos.html
  * FIXME Add links to xfs documentation!!!
  * CENTOS DHCP config: http://www.krizna.com/centos/install-configure-dhcp-server-centos-6/
  * :!: this is for NFS3. Would be better to use nfs4 and limit portmapper to NIS. CENTOS NFS config: http://www.malaya-digital.org/setup-a-minimal-centos-6-64-bit-nfs-server/

==== Software related ====

  *CernVM home: http://cernvm.cern.ch/portal/
  * Cervm-fs ATLAS (Courtesy of Anders): https://twiki.cern.ch/twiki/bin/view/Atlas/CernVMFS
    * https://twiki.cern.ch/twiki/bin/view/Atlas/Cvmfs21
    * https://twiki.cern.ch/twiki/bin/viewauth/Atlas/T2SquidDeployment
    * https://twiki.cern.ch/twiki/bin/view/CvmFS/StratumOnes
    * https://twiki.cern.ch/twiki/bin/view/Atlas/RPMCompatSLC6#Run_SLC5_64bit_binaries_on_SLC6
    * https://twiki.cern.ch/twiki/bin/view/Atlas/SLC6Readiness
    * SQUID setup:
      * https://twiki.cern.ch/twiki/bin/viewauth/Atlas/T2SquidDeployment
      * https://twiki.cern.ch/twiki/bin/view/Frontier/InstallSquid
      * https://twiki.cern.ch/twiki/bin/view/Frontier/WebHome
      * http://frontier.cern.ch/
    * Modules: a nice way of configuring cluster environments
      * http://modules.sourceforge.net/
      * an howto: http://www.admin-magazine.com/HPC/Articles/Environment-Modules
    * SLURM: a batch system.
      * http://www.schedmd.com/slurmdocs/quickstart_admin.html
    * MUNGE: a way of authenticating across nodes, needed by SLURM
      * https://code.google.com/p/munge/wiki/InstallationGuide

==== Node Deployment related ====
  * PXE installation: http://www.server-world.info/en/note?os=CentOS_6&p=pxe
  * tftp and selinux permission issue: http://serverfault.com/questions/106007/tftp-uploads-failing
  * Centos DNS config: 
    * http://ostechnix.wordpress.com/2013/01/25/setup-dns-server-step-by-step-in-centos-6-3-rhel-6-3-scientific-linux-6-3-3/
    * https://www.digitalocean.com/community/articles/how-to-install-the-bind-dns-server-on-centos-6
  * port forwarding with iptables: http://www.ridinglinux.org/2008/05/21/simple-port-forwarding-with-iptables-in-linux/
  * kickstarter and centos6:
    * http://www.openlogic.com/wazi/bid/188083/Automatic-CentOS-6-0-Installation-With-Kickstart
  * SystemImager (courtesy of Robert)
    * http://systemimager.sourceforge.net/
    * http://linuxdatacenter.blogspot.se/2010/05/systemimager-puppet.html
    * http://www.systemimager.org/documentation/systemimager-manual-4.1.6.pdf

===== Howtos =====

  * How-to use [[http://docs.saltstack.com/|salt]] to [[spice|spice]] things up
  * other [[iridium_cluster:howtos]] (admin access only)

===== Various other stuff =====

==== Wanted packages on nodes ====

[[iridium_cluster:wantedpackagesonnodes|click here to go to page (admin access only)]]
pfwiki

User Tools

Site Tools

Page Tools
