Setup xen-tools on Debian

by Pierre-Yves Landuré — last modified Jan 19, 2014 02:17

xen-tools ease and automates the creation of Xen domUs. This howto propose a standardized Debian GNU/Linux domU with backup infrastructure, easy SSH access and minimal security.

This howto is tested on:

Debian 6.0 Squeeze
Debian 7.0 Wheezy

Prerequisites

This howto needs :

a working outgoing SMTP server as described by Additions to a fresh Debian GNU/Linux install.
the Xen hypervisor as described by Install Xen on Debian.
the Backup Manager tool, as described by Install and setup Backup Manager on Debian.
the Logical Volume Manager (aka. LVM) (with a volume group setted up), as described by Setup and use the Logical Volume Manager (LVM) on Debian.

Parameters

Detect the dom0 IP address. In a NAT setup, the IP address is fixed to 10.0.0.127, but with a Bridge setup, the physical network interface IP address is used (here, eth0) :

MAINDOM_IP="10.0.0.127"
if [ $(command grep "^[^#]*bridge" '/etc/xen/xend-config.sxp' | command wc --lines) -gt 0 ]; then
  MAINDOM_IP="$(command ifconfig eth0 | command grep "inet " | command sed -e 's/^.*inet [^:]*:\([^ ]*\) .*$/\1/')"
fi

Check the detected IP address :

command echo "The IP address of the dom0 for the domUs is : ${MAINDOM_IP}"

Note : If the detected address is incorrect, provide the correct value :

MAINDOM_IP="192.168.1.5"

Installation

Environment preparation

Install the needed software :

command apt-get install xen-tools

Create the dummy0 interface if necessary :

if [ -z "$(command ifconfig | command grep "${MAINDOM_IP}")" ]; then
  MAINDOM_RANGE="$(command echo ${MAINDOM_IP} | command sed -e 's/\(.*\)\.[0-9]*/\1/')"
  command modprobe dummy
  command echo "dummy" >> '/etc/modules'
  command echo "
# Xen main interface.
auto dummy0
iface dummy0 inet static
    address ${MAINDOM_IP}
    netmask 255.255.255.0
    network ${MAINDOM_RANGE}.0
    broadcast ${MAINDOM_RANGE}.255" \
  >> '/etc/network/interfaces'
  command ifup dummy0
fi

Setting up the backup infrastructure

This howto setup a backup infrastructure based on Backup Manager. The domUs uploads the backups in the "/var/archives" folder of the dom0. Make sure this folder exists :

command mkdir --parents '/var/archives'
command chown root:backup '/var/archives'
command chmod ug+rwx '/var/archives'

Setup Backup Manager to open the '/var/archives' folder to virtual hosts, and to purge the domU's archives with the dom0 ones :

command sed -i -e 's/[#]*\(.*BM_REPOSITORY_GROUP=\).*$/\1"backup"/' \
            -e 's/[#]*\(.*BM_REPOSITORY_CHMOD=\).*$/\1"770"/' \
            -e 's/[#]*\(.*BM_ARCHIVE_STRICTPURGE=\).*$/\1"false"/' \
         '/etc/backup-manager.conf'

Create the xen-backup user, used by the domUs to upload the backups on the dom0 :

command adduser --system --shell '/bin/sh' --home '/var/lib/xen-backup' --disabled-password 'xen-backup'

Add the user to backup and ssh-users groups :

command adduser 'xen-backup' 'backup'
if [ -n "$(command grep 'ssh-users' '/etc/group')" ]; then
  command adduser 'xen-backup' 'ssh-users'
fi

Create a RSA key for passwordless authentication to the xen-backup account :

command mkdir --parents '/var/lib/xen-backup/.ssh/'
command ssh-keygen -t rsa -C "upgrade account key" -N "" -f '/var/lib/xen-backup/.ssh/id_rsa'
command cat '/var/lib/xen-backup/.ssh/id_rsa.pub' \
    >> '/var/lib/xen-backup/.ssh/authorized_keys'
command chown -R xen-backup:nogroup '/var/lib/xen-backup/.ssh'

Allow access to the SSH server by the domUs if Knockd is present :

SSH_PORT="$(command grep '^Port' '/etc/ssh/sshd_config' \
    | command sed -e 's/^Port[\t ]*//g')"
if [ -e '/etc/network/if-up.d/iptables-ssh-reject' ]; then
  command echo "#"\!"/bin/bash
# Allow SSH access from domU
command iptables -C INPUT ! -s 10.0.0.0/24 -p tcp -m state --state NEW --dport ${SSH_PORT} -j DROP > '/dev/null' \\
  || command iptables -A INPUT ! -s 10.0.0.0/24 -p tcp -m state --state NEW --dport ${SSH_PORT} -j DROP" \
   > '/etc/network/if-up.d/iptables-ssh-reject'
  command chmod +x '/etc/network/if-up.d/iptables-ssh-reject'
  while command iptables -D INPUT -p tcp -m state --state NEW --dport ${SSH_PORT} -j DROP; do echo; done
  /etc/network/if-up.d/iptables-ssh-reject
fi

SMTP server setup

Warning: the following instructions can break the dom0 setup. If the Exim 4 setup is customized, and the Xen network mode is NAT, make sure that the SMTP serveur listen on address 10.0.0.127 and accept to relay email from 10.0.0.0/24.

Setup the SMTP server to relay emails sent by domUs:

MAINDOM_RANGE="$(command echo ${MAINDOM_IP} | command sed -e 's/\(.*\)\.[0-9]*/\1/')"
command sed -i \
    -e "s|dc_local_interfaces=.*\$|dc_local_interfaces='127.0.0.1;${MAINDOM_IP}'|" \
    -e "s|dc_relay_nets=.*\$|dc_relay_nets='${MAINDOM_RANGE}.0/24'|" \
  '/etc/exim4/update-exim4.conf.conf'

Update the Exim 4 configuration :

command update-exim4.conf
/etc/init.d/exim4 restart

Configuration

Disk usage

To be safe, raise the created domU swap size to 1GB :

command sed -i -e 's/^\(swap[ ]*=[ ]*\)[^ ]*\([ ]*#.*\)$/\11Gb\2/' '/etc/xen-tools/xen-tools.conf'

Note : If there is poor available disk space, you may divide by 2 the size of swap and root partitions (from 1GB to 512MB and from 4GB to 2GB) :

command sed -i -e 's/^\(size[ ]*=[ ]*\)[^ ]*\([ ]*#.*\)$/\12Gb\2/' \
            -e 's/^\(swap[ ]*=[ ]*\)[^ ]*\([ ]*#.*\)$/\1512Mb\2/' \
         '/etc/xen-tools/xen-tools.conf'

Network parameters

When Xen use a NAT network, setup the created domUs network parameters :

if [ $(command grep '^[^#]*nat' '/etc/xen/xend-config.sxp' | command wc --lines) -gt 0 ]; then
  command sed -i \
      -e 's/^[# ]*\(gateway[ ]*=\).*$/\1 10.0.0.128/' \
      -e 's/^[# ]*\(netmask[ ]*=\).*$/\1 255.255.255.0/' \
    '/etc/xen-tools/xen-tools.conf'
fi

Note : if Xen use a bridged network, manually setup the local network parameters in "/etc/xen-tools/xen-tools.conf".

Using logical volumes

Set up xen-tools to create domUs disks with LVM. Detect the first system Volume Group :

LVM_VG="$(command vgdisplay \
  | command grep "VG Name" \
  | command head --lines=1 \
  | command cut --characters=25-)"

Apply the setting :

command sed -i -e "s/^# \(lvm = \).*/\1${LVM_VG}/" '/etc/xen-tools/xen-tools.conf'

Kernel selection

Setup the domU kernel automatic detection :

command sed -i -e 's/^\(kernel =\).*\$/\1 \/boot\/vmlinuz-`uname -r`/' \
            -e 's/^\(initrd =\).*\$/\1 \/boot\/initrd.img-`uname -r`/' \
            '/etc/xen-tools/xen-tools.conf'

Fixing "clocksource/0: Time went backwards"

To prevent a domU crash after dom0 reboot with "clocksource/0: Time went backwards" error, add "extra = 'clocksource=xen'" option to domU configuration template :

command sed -i -e "/^on_crash/a\\
\\
# Preventing clocksource/0: Time went backwards\\
extra = 'clocksource=xen'" \
    '/etc/xen-tools/xm.tmpl'

domU initialization

Create a role initializing domUs with:

Installation of packages locales, sudo, ntp, cron-apt, rkhunter, vim, screen, backup-manager and fail2ban
Automated upgrades.
Backup infrastructure based on backup-manager.

Passwordless access to domU

Create a RSA key for the dom0 root account if you want to easily login in domU root account via SSH :

if [ ! -e "${HOME}/.ssh/id_rsa" ]; then
  command ssh-keygen -t rsa -f "${HOME}/.ssh/id_rsa"
fi

Automated upgrades

Create a RSA key for passwordless authentication to the "xen-upgrade" account of the domU :

command mkdir --parents '/etc/xen-tools/ssh-keys'
command ssh-keygen -t rsa -C "upgrade account key" -N "" -f '/etc/xen-tools/ssh-keys/xen-upgrade-rsa'

Install the domU automated upgrade tool :

command wget 'https://raw.github.com/biapy/howto.biapy.com/master/xen-tools/xen-apt-get' \
    --no-check-certificate --output-document='/usr/bin/xen-apt-get'
command chmod +x '/usr/bin/xen-apt-get'

Upgrade all of active domUs with :

# command xen-apt-get update
# command xen-apt-get upgrade

Role setup

Set up the configuration file for the role "automatic" :

command wget 'https://raw.github.com/biapy/howto.biapy.com/master/xen-tools/automatic' \
    --no-check-certificate --output-document='/etc/xen-tools/role.d/automatic'
command chmod +x '/etc/xen-tools/role.d/automatic'

Create a skeleton for domU initialization :

command mkdir --parents '/etc/xen-tools/skel/root'
command mkdir --parents '/etc/xen-tools/skel/root/.ssh'
command mkdir --parents '/etc/xen-tools/skel/etc/default'
command mkdir --parents '/etc/xen-tools/skel/etc/exim4'
command mkdir --parents '/etc/xen-tools/skel/etc/cron.d'
command mkdir --parents '/etc/xen-tools/skel/etc/cron.daily'
command mkdir --parents '/etc/xen-tools/skel/etc/apt/apt.conf.d'
command mkdir --parents '/etc/xen-tools/skel/etc/xen-data'
command mkdir --parents '/etc/xen-tools/skel/usr/bin'

Fill the skeleton with dom0 configuration :

command cp '/root/.bashrc' '/etc/xen-tools/skel/root/'
if [ -e /root/.vimrc ]; then
  command cp '/root/.vimrc' '/etc/xen-tools/skel/root/'
fi
command cp '/etc/timezone' '/etc/xen-tools/skel/etc/'
command cp '/etc/localtime' '/etc/xen-tools/skel/etc/'
command cp '/etc/locale.gen' '/etc/xen-tools/skel/etc/'
command cp '/etc/environment' '/etc/xen-tools/skel/etc/'
command cp '/etc/default/locale' '/etc/xen-tools/skel/etc/default/'

Add the dom0's SSH public key to domU known hosts :

command ssh-keyscan -H -t rsa "${MAINDOM_IP}" >> '/etc/xen-tools/skel/root/.ssh/known_hosts'

Allow the passwordless authentication to the domU root account:

if [ -e "${HOME}/.ssh/id_rsa.pub" ]; then
    command cat "${HOME}/.ssh/id_rsa.pub" \
        >> "/etc/xen-tools/skel/root/.ssh/authorized_keys"
fi

Setup the redirection of emails sent by domU to dom0 SMTP server :

command wget 'https://raw.github.com/biapy/howto.biapy.com/master/xen-tools/update-exim4.conf.conf' \
    --output-document='/etc/xen-tools/skel/etc/exim4/update-exim4.conf.conf'
command sed -i -e "s/dc_smarthost=.*\$/dc_smarthost='${MAINDOM_IP}'/" \
         '/etc/xen-tools/skel/etc/exim4/update-exim4.conf.conf'
echo "root@$(command hostname --fqdn)" > '/etc/xen-tools/skel/root/.email'

Add the Backup Manager administration tool to domU :

command wget 'https://raw.github.com/biapy/howto.biapy.com/master/backup-manager/backup-manager-tools' \
    --quiet --no-check-certificate --output-document='/etc/xen-tools/skel/usr/bin/backup-manager-tools'
command chmod +x '/etc/xen-tools/skel/usr/bin/backup-manager-tools'

Download Backup Manager configuration for domU :

command wget 'https://raw.github.com/biapy/howto.biapy.com/master/xen-tools/backup-manager.conf' \
    --no-check-certificate --output-document='/etc/xen-tools/skel/etc/backup-manager.conf'
command sed -i -e "s|[#]*\(.*BM_UPLOAD_SSH_HOSTS=\).*$|\1\"${MAINDOM_IP}\"|" \
         '/etc/xen-tools/skel/etc/backup-manager.conf'

Add the cron setup for Backup Manager to the skeleton :

command echo '#!/bin/sh
# cron script for backup-manager
test -x /usr/sbin/backup-manager || exit 0
/usr/sbin/backup-manager' \
    > '/etc/xen-tools/skel/etc/cron.daily/backup-manager'
command chmod +x '/etc/xen-tools/skel/etc/cron.daily/backup-manager'

Setup the dom0 SSH port :

SSH_PORT="$(command grep '^Port' '/etc/ssh/sshd_config' \
    | command sed -e 's/^Port[\t ]*//g')"
command sed -i \
    -e "s/[#]*\(.*BM_UPLOAD_SSH_PORT=\).*\$/\1\"${SSH_PORT}\"/" \
  '/etc/xen-tools/skel/etc/backup-manager.conf'

Add to the skeleton the private RSA key for dom0's xen-backup account passwordless authentication :

command cp '/var/lib/xen-backup/.ssh/id_rsa' '/etc/xen-tools/skel/etc/xen-data/'

Remerciements

Thanks to xen-tools developers.

Thanks to Xen Networking authors on Xen wiki.
Thanks to GaB for Manage SSH known_hosts entries.