Grid Installation Guide
Contents
Contents
Currently, the packages are only available on SL5/SL6 and CentosOS7.
Installation of SL6
Here you have 2 options:
- Installation natively on a machine
- Installation on a virtual machine
In this example, I will explain the procedure for the later option, on a virtual machine. The procedure is exactly the same for a native installation of course.
Installation on a virtual machine
In this case, the native platform is Ubuntu LTS 16.04 (x86_64) with kernel 4.4.0-22-generic. Ubuntu LTS is a better alternative as it is supported for quite some time and also DESY native operating system for desktop is going towards Ubuntu. First, you need to install a virtual machine software.
There are quite some around but the most famous might be Oracle VirtualBox (https://www.virtualbox.org/wiki/Linux_Downloads). The installation is quite simple and all explained.
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add - sudo apt-get update sudo apt-get install virtualbox-5.0 virtualbox-dkms virtualbox-guest-*
Then virtual box must be installed.
The next step is to install SL6 (RHEL6) as a virtual machine. For that, open VirtualBox and click on New. Choose the operating system (linux - Other linux 64 bits) and name it SL6.
Assign 1024 MB of memory minimum (depending how you want to install SL6 as minimal or with the full desktop environment). Create a Hard Drive VDI of 8 GB minimum (Dynamically allocated). At the end you should see the new machine created, first we need to adjust few parameters and then install the new operating system.
Go to settings and change in system :
- Motherboard chipset to ICH9
- Processor : 2
- Graphic : 128 MB
Now, let's install SL6. You can get the latest iso from the SL6 website (https://www.scientificlinux.org/downloads/sl-versions/sl6/). The latest current is 6.7 (August 26, 2015). Once the iso is downloaded. In the VM, in settings, go to storage and select the optical drive. Next to the attributes, there is a small CD icon, click on it and then select the SL6 iso file. When you will boot for the first time, it will then boot on the iso file.
SL6 Installation
The installation is fairly simple. At the boot menu, select 'Install system with basic video driver'. Skip the media check. Select the language, keyboard layout. Select fresh install.
At the hostname choice, enter a hostname for the PC (I would advice the hostname of the host PC + VirtualBox, i.e. flchcallabX-VirtualBox). On the bottom left corner, click on 'Configure Network'. It will pop a windows, select the connection that appears and click 'Edit'. On the top then select 'Connect automatically' If you don t do that, then it will not automatically connect on boot..
Then select the Time zone, enter a root password and select a partition scheme ("Use All space"). After partitionning, the user can choose different install mode for the operating system (Desktop, Minimal, Server...).
For this example, I choose Basic Server and click on Customize now. In Desktop, click on Font, Legacy X Window System compatibility, Remote Desktop Clients and X Window System. Click ok and then it will start to install SL6.
At the end, reboot.
SL6 Configuration
At the first boot, a new user needs to be added. Connect as root and enter :
useradd calice passwd calice
This will add a new user calice. Now we need to give sudo rights to this new user.
visudo
Find the line :
## Allow root to run any command anywhere root ALL=(ALL) ALL
And add :
## Allow root to run any command anywhere root ALL=(ALL) ALL calice ALL=(ALL) ALL
Now we can disconnect from the root account and connect to the new calice account with sudo rights.
First thing update the package repository :
sudo yum update sudo yum upgrade
To make things easier in the following, I advice to see the ssh server configuration part
Then now we need to activate the EPEL repository. For that type : (for ## RHEL/CentOS 6 64-Bit ##)
wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm sudo rpm -ivh epel-release-6-8.noarch.rpm sudo yum update
EMI packages/repo
A nice tutorial is available here : https://twiki.cern.ch/twiki/bin/view/EMI/GenericInstallationConfigurationEMI3
This section is specific on the part for the grid ui and tools installation. EMI is the main middleware available providing all the tools necessary for the grid.
- Certificates installation
Install this repo in /etc/yum.repos.d/ Then run :
sudo yum update sudo yum install ca-policy-egi-core
- EMI Repository
We need now to install the EMI repository. Get the key first : http://emisoft.web.cern.ch/emisoft/dist/EMI/3/RPM-GPG-KEY-emi by doing
sudo rpm --import http://emisoft.web.cern.ch/emisoft/dist/EMI/3/RPM-GPG-KEY-emi
Install yum-priorities package and modify /etc/yum/pluginconf.d/priorities.conf like that
[main] enabled = 1 check_obsoletes = 1
Then do
wget http://emisoft.web.cern.ch/emisoft/dist/EMI/3/sl6/x86_64/base/emi-release-3.0.0-2.el6.noarch.rpm sudo yum localinstall emi-release-3.0.0-2.el6.noarch.rpm sudo yum update sudo yum install emi-ui
if you get a problem of conflics with openssl.sl6 and openssl.el6
sudo yum downgrade openssl sudo yum install openssl-devel mysql-devel sudo yum install emi-ui
Then the grid tools are installed. A few more tweaking is needed in order to make things working.
We need to download the vomses configuration files. For this, do :
sudo scp -r username@nafhh-ilc02.desy.de:/afs/desy.de/group/grid/www/html/etc/vomses/ /etc/ sudo rm /etc/vomses/vomses.save sudo scp -r username@nafhh-ilc02.desy.de:/afs/desy.de/group/grid/www/html/etc/grid-security/vomsdir/ /etc/grid-security/
Then you need to make a script like that :
echo "+" echo "+ Initializing grid on SLD6" echo "+" export GRID_SECURITY_DIR=/etc/grid-security export X509_CERT_DIR=${GRID_SECURITY_DIR}/certificates export X509_VOMS_DIR=${GRID_SECURITY_DIR}/vomsdir export VOMS_USERCONF=/etc/vomses/vomses export X509_USER_PROXY=$HOME/k5-ca-proxy.pem export GLOBUS_LOCATION=/usr export SRM_PATH=/usr/share/srm export PERL5LIB=/usr/share/perl5 export PYTHONPATH=${PYTHONPATH}:/usr/lib64/python2.6/site-packages:/usr/lib/python2.6/site-packages export PATH=/usr/sbin:$PATH #voms-proxy-init -debug voms-proxy-init -verify -voms calice:/calice/Role=production # test if libpdcap is already preloaded echo $LD_PRELOAD | grep libpdcap > /dev/null if [ $? -eq 1 ]; then for f in /usr/lib64/libpdcap.so; do if [ -e $f ] ; then export LD_PRELOAD=$LD_PRELOAD:$f break fi done # libpdcap.so need libgsiTunnel.so since ilc_v01-17 on pal and NAF2. for f in /usr/lib64/dcap/libgsiTunnel.so; do if [ -e $f ] ; then export DCACHE_IO_TUNNEL=$f break fi done fi export DCACHE_CLIENT_ACTIVE=1 export LCG_CATALOG_TYPE=lfc export LFC_HOST=grid-lfc.desy.de
Call it start_proxy.sh and do chmod +x start_proxy.sh .
Last step is the installation of your grid certificate in /home/calice/.globus/. For that, there is tutorial here : http://grid.desy.de/certs/ Once this is done, to check if it is working do :
source start_proxy.sh + + Initializing grid on SLD6 + Enter GRID pass phrase for this identity: Contacting grid-voms.desy.de:15102 [/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de] "calice"... Remote VOMS server contacted succesfully. Created proxy in /home/calice/k5-ca-proxy.pem. Your proxy is valid until Tue May 31 03:36:10 CEST 2016
To check if the grid tools are working do :
gfal-ls -l srm://dcache-se-desy.desy.de/pnfs/desy.de/calice/tb-desy/
dCache-GUI
The installation procedure is available here : https://github.com/ebrianne/dCache-gui This small GUI is used to upload Testbeam data directly on the dCache from the Testbeam Hut.
In Construction
For this part, you need a valid grid certificate and the right to upload data on the dCache for CALICE (need the production role in the calice-vo). For getting the production role, send a request via https://grid-voms.desy.de:8443/voms/calice
Packages to install in addition
Packages :
- bash-completion
sudo yum install bash-completion
- qt5
sudo yum install qt5-*
- g++ 4.8
sudo wget -O /etc/yum.repos.d/slc6-devtoolset.repo http://linuxsoft.cern.ch/cern/devtoolset/slc6-devtoolset.repo sudo yum install devtoolset-2 echo "source /opt/rh/devtoolset-2/enable" > /home/calice/.bashrc
- svn
sudo yum install subversion
- git
sudo yum install git
- emacs
sudo yum install emacs
- sshfs
sudo yum install sshfs sudo usermod -a -G fuse calice sudo reboot
- doxygen
sudo yum install doxygen
- cmake
sudo yum install cmake
Optional
Configure ssh server
You can configure the ssh server in order to connect with a terminal from the host to the virtual machine. In SL6, modify the file in /etc/ssh/sshd_config. Find the line "Port 22" and replace the port number with any port > 1000 (i.e. 5622). Then you need to open the firewall to that port :
sudo iptables -I INPUT -p tcp -m tcp --dport 5622 -j ACCEPT sudo service iptables save sudo service iptables restart
Then you need to configure the NAT in VirtualBox. On your VM, go to settings -> Network. Click on advanced and Port Configuration. Add a new rule :
Name |
Protocol |
Host IP |
Host Port |
Guest IP |
Guest Port |
ssh |
TCP |
|
5622 |
|
5622 |
Now you can connect :
ssh -X -p 5622 calice@localhost
Configure VM to boot automatically and shutdown gracefully with the host
With Ubuntu 16.04, the service are started with systemd. You can make a systemd script that commands the virtual machine to start at boot and shutdown correctly with the host is turned off.
Create a script in /etc/systemd/system/ named vboxvmservice@.service
[Unit] Description=VBox Virtual Machine %i Service Requires=virtualbox.service After=virtualbox.service [Service] User=calice Group=vboxusers ExecStart=/usr/bin/VBoxHeadless -s %i ExecStop=/usr/bin/VBoxManage controlvm %i acpipowerbutton ExecStopPost=sleep 20 [Install] WantedBy=multi-user.target
Then type :
sudo usermod -a -G vboxusers calice sudo reboot
sudo systemctl enable vboxvmservice@SL6.service sudo systemctl start vboxvmservice@SL6.service sudo systemctl status vboxvmservice@SL6.service
You should get
● vboxvmservice@SL6.service - VBox Virtual Machine SL6 Service Loaded: loaded (/etc/systemd/system/vboxvmservice@.service; enabled; vendor preset: enabled) Active: active (running) since Mo 2016-05-30 11:53:25 CEST; 4h 32min ago Main PID: 1487 (VBoxHeadless) CGroup: /system.slice/system-vboxvmservice.slice/vboxvmservice@SL6.service ├─1487 /usr/lib/virtualbox/VBoxHeadless -s SL6 ├─1651 /usr/lib/virtualbox/VBoxXPCOMIPCD ├─1708 /usr/lib/virtualbox/VBoxSVC --auto-shutdown └─1937 /usr/bin/pulseaudio --start --log-target=syslog Mai 30 11:53:25 flchcallab5 systemd[1]: Started VBox Virtual Machine SL6 Service. Mai 30 11:53:33 flchcallab5 pulseaudio[1937]: [pulseaudio] server-lookup.c: Unable to contact D-Bus Mai 30 11:53:33 flchcallab5 pulseaudio[1937]: [pulseaudio] main.c: Unable to contact D-Bus: org.fre Mai 30 11:53:40 flchcallab5 pulseaudio[1937]: [pulseaudio] bluez5-util.c: GetManagedObjects() faile
Mounting Host Home folder in VM
One thing practical is the auto-mounting of the home folder of the host in the guest. For this, VirtualBox provides all the tools. First, the virtual guest additions needs to be installed. Get the iso file here : http://download.virtualbox.org/virtualbox/5.0.16/VBoxGuestAdditions_5.0.16.iso Mount the cd in virtualbox and reboot the guest. Type :
sudo mkdir /cdrom sudo mount /dev/cdrom /cdrom sudo yum install kernel-devel cd /cdrom sudo ./VBoxLinuxAdditions.run sudo shutdown -h now
After the shutdown, we can configure VBox to have a share folder. Go to shared folders and Add a new shared folder. Click on Auto-mount, Make permanent and Read-only if you only want to have read access only.
Boot the guest, log in and type :
sudo usermod -a -G vboxsf calice
Logout and enjoy the shared folder in /media/sf_XXXX You can do a symlink to it if you want. ln -s /media/sf_XXXX <link>
To make a sshfs readable from the host for the guest, you need to allow it. On the host modify /etc/fuse.conf and uncomment the line :
# Allow non-root users to specify the allow_other or allow_root mount options. user_allow_other
When you do sshfs then now specify "sshfs -oallow_other ..." this will make the reading from the guest possible.