Grid Installation Guide

Contents

Contents

  1. Grid Installation Guide
    1. Installation of SL6
      1. Installation on a virtual machine
      2. SL6 Installation
      3. SL6 Configuration
    2. EMI packages/repo
    3. dCache-GUI
    4. Packages to install in addition
    5. Optional
      1. Configure ssh server
      2. Configure VM to boot automatically and shutdown gracefully with the host
      3. Mounting Host Home folder in VM

/!\ Currently, the packages are only available on SL5/SL6 and CentosOS7. /!\

Installation of SL6

Here you have 2 options:

In this example, I will explain the procedure for the later option, on a virtual machine. The procedure is exactly the same for a native installation of course.

Installation on a virtual machine

In this case, the native platform is Ubuntu LTS 16.04 (x86_64) with kernel 4.4.0-22-generic. Ubuntu LTS is a better alternative as it is supported for quite some time and also DESY native operating system for desktop is going towards Ubuntu. First, you need to install a virtual machine software.

There are quite some around but the most famous might be Oracle VirtualBox (https://www.virtualbox.org/wiki/Linux_Downloads). The installation is quite simple and all explained. 

wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
sudo apt-get update
sudo apt-get install virtualbox-5.0 virtualbox-dkms virtualbox-guest-*

Then virtual box must be installed.

VirtualBox.png

The next step is to install SL6 (RHEL6) as a virtual machine. For that, open VirtualBox and click on New. Choose the operating system (linux - Other linux 64 bits) and name it SL6.

CreateVM.png

Assign 1024 MB of memory minimum (depending how you want to install SL6 as minimal or with the full desktop environment). Create a Hard Drive VDI of 8 GB minimum (Dynamically allocated). At the end you should see the new machine created, first we need to adjust few parameters and then install the new operating system.

CreateVM2.png CreateVM3.png

Go to settings and change in system :

CreateVM4.png

Now, let's install SL6. You can get the latest iso from the SL6 website (https://www.scientificlinux.org/downloads/sl-versions/sl6/). The latest current is 6.7 (August 26, 2015). Once the iso is downloaded. In the VM, in settings, go to storage and select the optical drive. Next to the attributes, there is a small CD icon, click on it and then select the SL6 iso file. When you will boot for the first time, it will then boot on the iso file.

CreateVM5.png SL6Boot.png

SL6 Installation

The installation is fairly simple. At the boot menu, select 'Install system with basic video driver'. Skip the media check. Select the language, keyboard layout. Select fresh install.

InstallSL6.png

At the hostname choice, enter a hostname for the PC (I would advice the hostname of the host PC + VirtualBox, i.e. flchcallabX-VirtualBox). On the bottom left corner, click on 'Configure Network'. It will pop a windows, select the connection that appears and click 'Edit'. On the top then select 'Connect automatically' /!\ If you don t do that, then it will not automatically connect on boot.. /!\

NetworkHostname.png

Then select the Time zone, enter a root password and select a partition scheme ("Use All space"). After partitionning, the user can choose different install mode for the operating system (Desktop, Minimal, Server...).

InstallMode.png

For this example, I choose Basic Server and click on Customize now. In Desktop, click on Font, Legacy X Window System compatibility, Remote Desktop Clients and X Window System. Click ok and then it will start to install SL6.

BSCustomize.png SL6Installing.png

At the end, reboot.

SL6 Configuration

At the first boot, a new user needs to be added. Connect as root and enter : 

useradd calice
passwd calice

This will add a new user calice. Now we need to give sudo rights to this new user. 

visudo

Find the line : 

## Allow root to run any command anywhere
root     ALL=(ALL)      ALL

And add : 

## Allow root to run any command anywhere
root     ALL=(ALL)      ALL
calice   ALL=(ALL)      ALL

Now we can disconnect from the root account and connect to the new calice account with sudo rights.

First thing update the package repository : 

sudo yum update
sudo yum upgrade

To make things easier in the following, I advice to see the ssh server configuration part

Then now we need to activate the EPEL repository. For that type : (for ## RHEL/CentOS 6 64-Bit ##) 

wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
sudo rpm -ivh epel-release-6-8.noarch.rpm
sudo yum update

EMI packages/repo

A nice tutorial is available here : https://twiki.cern.ch/twiki/bin/view/EMI/GenericInstallationConfigurationEMI3

This section is specific on the part for the grid ui and tools installation. EMI is the main middleware available providing all the tools necessary for the grid.

Install this repo in /etc/yum.repos.d/ Then run : 

sudo yum update
sudo yum install ca-policy-egi-core

We need now to install the EMI repository. Get the key first : http://emisoft.web.cern.ch/emisoft/dist/EMI/3/RPM-GPG-KEY-emi by doing 

sudo rpm --import http://emisoft.web.cern.ch/emisoft/dist/EMI/3/RPM-GPG-KEY-emi

Install yum-priorities package and modify /etc/yum/pluginconf.d/priorities.conf like that 

[main]
enabled = 1
check_obsoletes = 1

Then do 

wget http://emisoft.web.cern.ch/emisoft/dist/EMI/3/sl6/x86_64/base/emi-release-3.0.0-2.el6.noarch.rpm
sudo yum localinstall emi-release-3.0.0-2.el6.noarch.rpm
sudo yum update
sudo yum install emi-ui

/!\ if you get a problem of conflics with openssl.sl6 and openssl.el6 /!\ 

sudo yum downgrade openssl
sudo yum install openssl-devel mysql-devel
sudo yum install emi-ui

Then the grid tools are installed. A few more tweaking is needed in order to make things working.

We need to download the vomses configuration files. For this, do : 

sudo scp -r username@nafhh-ilc02.desy.de:/afs/desy.de/group/grid/www/html/etc/vomses/ /etc/
sudo rm /etc/vomses/vomses.save
sudo scp -r username@nafhh-ilc02.desy.de:/afs/desy.de/group/grid/www/html/etc/grid-security/vomsdir/ /etc/grid-security/

Then you need to make a script like that : 

echo "+"
echo "+ Initializing grid on SLD6"
echo "+"

export GRID_SECURITY_DIR=/etc/grid-security
export X509_CERT_DIR=${GRID_SECURITY_DIR}/certificates
export X509_VOMS_DIR=${GRID_SECURITY_DIR}/vomsdir
export VOMS_USERCONF=/etc/vomses/vomses
export X509_USER_PROXY=$HOME/k5-ca-proxy.pem
export GLOBUS_LOCATION=/usr
export SRM_PATH=/usr/share/srm
export PERL5LIB=/usr/share/perl5
export PYTHONPATH=${PYTHONPATH}:/usr/lib64/python2.6/site-packages:/usr/lib/python2.6/site-packages
export PATH=/usr/sbin:$PATH

#voms-proxy-init -debug
voms-proxy-init -verify -voms calice:/calice/Role=production

# test if libpdcap is already preloaded
echo $LD_PRELOAD | grep libpdcap > /dev/null
if [ $? -eq 1 ]; then
    for f in /usr/lib64/libpdcap.so; do
        if [ -e $f ] ; then
            export LD_PRELOAD=$LD_PRELOAD:$f
            break
        fi
    done
# libpdcap.so need libgsiTunnel.so since ilc_v01-17 on pal and NAF2.
    for f in /usr/lib64/dcap/libgsiTunnel.so; do
        if [ -e $f ] ; then
            export DCACHE_IO_TUNNEL=$f
            break
        fi
    done
fi

export DCACHE_CLIENT_ACTIVE=1
export LCG_CATALOG_TYPE=lfc
export LFC_HOST=grid-lfc.desy.de

Call it start_proxy.sh and do  chmod +x start_proxy.sh .

Last step is the installation of your grid certificate in /home/calice/.globus/. For that, there is tutorial here : http://grid.desy.de/certs/ Once this is done, to check if it is working do : 

source start_proxy.sh

+
+ Initializing grid on SLD6
+
Enter GRID pass phrase for this identity:
Contacting grid-voms.desy.de:15102 [/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de] "calice"...
Remote VOMS server contacted succesfully.


Created proxy in /home/calice/k5-ca-proxy.pem.

Your proxy is valid until Tue May 31 03:36:10 CEST 2016

To check if the grid tools are working do : 

gfal-ls -l srm://dcache-se-desy.desy.de/pnfs/desy.de/calice/tb-desy/

dCache-GUI

The installation procedure is available here : https://github.com/ebrianne/dCache-gui This small GUI is used to upload Testbeam data directly on the dCache from the Testbeam Hut.

/!\ In Construction /!\

For this part, you need a valid grid certificate and the right to upload data on the dCache for CALICE (need the production role in the calice-vo). For getting the production role, send a request via https://grid-voms.desy.de:8443/voms/calice

Packages to install in addition

Packages : 

sudo yum install bash-completion

sudo yum install qt5-*

sudo wget -O /etc/yum.repos.d/slc6-devtoolset.repo http://linuxsoft.cern.ch/cern/devtoolset/slc6-devtoolset.repo
sudo yum install devtoolset-2
echo "source /opt/rh/devtoolset-2/enable" > /home/calice/.bashrc

sudo yum install subversion

sudo yum install git

sudo yum install emacs

sudo yum install sshfs
sudo usermod -a -G fuse calice
sudo reboot

sudo yum install doxygen

sudo yum install cmake

Optional

Configure ssh server

You can configure the ssh server in order to connect with a terminal from the host to the virtual machine. In SL6, modify the file in /etc/ssh/sshd_config. Find the line "Port 22" and replace the port number with any port > 1000 (i.e. 5622). Then you need to open the firewall to that port : 

sudo iptables -I INPUT -p tcp -m tcp --dport 5622 -j ACCEPT
sudo service iptables save 
sudo service iptables restart

Then you need to configure the NAT in VirtualBox. On your VM, go to settings -> Network. Click on advanced and Port Configuration. Add a new rule :

Name

Protocol

Host IP

Host Port

Guest IP

Guest Port

ssh

TCP

5622

5622

NAT.png NAT2.png

Now you can connect : 

ssh -X -p 5622 calice@localhost

Configure VM to boot automatically and shutdown gracefully with the host

With Ubuntu 16.04, the service are started with systemd. You can make a systemd script that commands the virtual machine to start at boot and shutdown correctly with the host is turned off.

Create a script in /etc/systemd/system/ named vboxvmservice@.service 

[Unit]
Description=VBox Virtual Machine %i Service
Requires=virtualbox.service
After=virtualbox.service

[Service]
User=calice
Group=vboxusers
ExecStart=/usr/bin/VBoxHeadless -s %i
ExecStop=/usr/bin/VBoxManage controlvm %i acpipowerbutton
ExecStopPost=sleep 20

[Install]
WantedBy=multi-user.target

Then type : 

sudo usermod -a -G vboxusers calice
sudo reboot

sudo systemctl enable vboxvmservice@SL6.service
sudo systemctl start vboxvmservice@SL6.service
sudo systemctl status vboxvmservice@SL6.service

You should get 

● vboxvmservice@SL6.service - VBox Virtual Machine SL6 Service
   Loaded: loaded (/etc/systemd/system/vboxvmservice@.service; enabled; vendor preset: enabled)
   Active: active (running) since Mo 2016-05-30 11:53:25 CEST; 4h 32min ago
 Main PID: 1487 (VBoxHeadless)
   CGroup: /system.slice/system-vboxvmservice.slice/vboxvmservice@SL6.service
           ├─1487 /usr/lib/virtualbox/VBoxHeadless -s SL6
           ├─1651 /usr/lib/virtualbox/VBoxXPCOMIPCD
           ├─1708 /usr/lib/virtualbox/VBoxSVC --auto-shutdown
           └─1937 /usr/bin/pulseaudio --start --log-target=syslog

Mai 30 11:53:25 flchcallab5 systemd[1]: Started VBox Virtual Machine SL6 Service.
Mai 30 11:53:33 flchcallab5 pulseaudio[1937]: [pulseaudio] server-lookup.c: Unable to contact D-Bus
Mai 30 11:53:33 flchcallab5 pulseaudio[1937]: [pulseaudio] main.c: Unable to contact D-Bus: org.fre
Mai 30 11:53:40 flchcallab5 pulseaudio[1937]: [pulseaudio] bluez5-util.c: GetManagedObjects() faile

Mounting Host Home folder in VM

One thing practical is the auto-mounting of the home folder of the host in the guest. For this, VirtualBox provides all the tools. First, the virtual guest additions needs to be installed. Get the iso file here : http://download.virtualbox.org/virtualbox/5.0.16/VBoxGuestAdditions_5.0.16.iso Mount the cd in virtualbox and reboot the guest. Type : 

sudo mkdir /cdrom
sudo mount /dev/cdrom /cdrom
sudo yum install kernel-devel
cd /cdrom
sudo ./VBoxLinuxAdditions.run
sudo shutdown -h now

After the shutdown, we can configure VBox to have a share folder. Go to shared folders and Add a new shared folder. Click on Auto-mount, Make permanent and Read-only if you only want to have read access only.

sharedfolder.png

Boot the guest, log in and type : 

sudo usermod -a -G vboxsf calice

Logout and enjoy the shared folder in /media/sf_XXXX You can do a symlink to it if you want.  ln -s /media/sf_XXXX <link> 

To make a sshfs readable from the host for the guest, you need to allow it. On the host modify /etc/fuse.conf and uncomment the line : 

# Allow non-root users to specify the allow_other or allow_root mount options.
user_allow_other

When you do sshfs then now specify "sshfs -oallow_other ..." this will make the reading from the guest possible.

CategoryHCAL

GridInstall (last edited 2016-12-21 15:09:39 by EldwanBrianne)