Install Xen on Debian

This how-to is tested on:

• Debian 6.0 Squeeze
• Debian 7.0 Wheezy

Vocabulary

Xen use specific terms to identify virtual hosts:

• Hypervisor: Xen is an hypervisor. In computing, a hypervisor or virtual machine monitor (VMM) is a piece of computer software, firmware or hardware that creates and runs virtual machines (source: Hypervisor on Wikipedia).
• Dom0: The Dom0, for Domain n°0, is the operating system that manage the Xen hypervisor. It is the original host where Xen has been installed.
• DomU: The DomU, for User Domains, are Xen's virtual hosts.

Parameters

Provide the amount of RAM assigned to the Dom0. The setup proposed on this site needs the Dom0 to have sufficient RAM to install the DomUs. Adjust the default value to your needs and the amount of RAM available on the server. Provide a value in MegaBytes:

DOM0_RAM=512

Provide the amount of CPU cores assigned definitively to the Dom0 to avoid a performance drop in case of heavy disk I/O load. One CPU core is a good default:

DOM0_CPU_COUNT=1

Xen has many network architecture options. This howto describe 2 simple ones:

• bridge: domUs are on the dom0 local network. Other network hosts can directly access the domUs services. This configuration is best used in local network setup.
• NAT: domUs are on a private local network. There are not available on dom0 network. This configuration is prefered when dom0 is directly connected to Internet (dedicated server, etc.)

Set up a NAT network (recommended):

XEN_NETWORK_MODE="nat"

Note: to setup a bridge network, use :

# XEN_NETWORK_MODE="bridge"

Installation

Environment preparation

Detect the available xen-utils version :

XEN_UTILS="$(command apt-cache pkgnames xen-utils)"

Install the Xen hypervisor and kernel adapted to the operating system architecture (32 or 64 bits) :

if [ "$(command uname --machine)" = "x86_64" ]; then
  if [ -n "$(command apt-cache pkgnames 'xen-linux-system-amd64')" ]; then
    command apt-get -y install xen-linux-system-amd64
  else
    command apt-get -y install xen-hypervisor-amd64 linux-image-2.6-xen-amd64 \
        linux-headers-2.6-xen-amd64 ${XEN_UTILS} bridge-utils
  fi
else
  if [ -n "$(command apt-cache pkgnames 'xen-linux-system-686-pae')" ]; then
    command apt-get install xen-linux-system-686-pae
  else
    command apt-get -y install xen-hypervisor-i386 linux-image-2.6-xen-686 \
        linux-headers-2.6-xen-686 ${XEN_UTILS} bridge-utils
  fi
fi

If the system boot manager is Grub 2 (grub-pc package), give Xen kernels the highest priority and disable the automatic OS discovery to prevent DomUs to be added to grub menu :

command dpkg-divert --divert '/etc/grub.d/05_linux_xen' --rename '/etc/grub.d/20_linux_xen'
echo '# Disable OS probing.
GRUB_DISABLE_OS_PROBER=true' >> '/etc/default/grub'

Add to Grub setup the options needed for Xen kernels :

if [ -e '/etc/default/grub' ]; then
  if [ -z "$(command grep 'GRUB_CMDLINE_XEN' '/etc/default/grub')" ]; then
    echo '
# Xen kernels configuration
GRUB_CMDLINE_XEN_DEFAULT=""
GRUB_CMDLINE_XEN=""' >> '/etc/default/grub'
  fi
fi

Update the available kernels list :

command update-grub

Reboot the system to start Xen hypervisor :

command reboot

Setting up initramfs

Make sure initramfs integrates the Xen modules (paticularly xen_blkfront, used by the domUs to see xvda devices):

command sed -i -e 's/^MODULES=.*/MODULES=most/' '/etc/initramfs-tools/initramfs.conf'
command sed -i -e 's/^MODULES=.*/MODULES=most/' '/etc/initramfs-tools/conf.d/'*

Update initramfs contents:

command update-initramfs -u

Note: the setting before fix the error 'ALERT! /dev/xvda2 does not exist. Dropping to a shell!' present when booting a domU created by xen-create-image. To fix this problem, copy the updated initrd file in the /boot folder of concerned domUs.

Optimization

Apply the setting to Xen configuration :

command sed -i -e "s/^[# ]*\((dom0-cpus\).*\().*\)\$/\1 ${DOM0_CPU_COUNT}\2/" \
         '/etc/xen/xend-config.sxp'

Force the Dom0 to only use the CPU cores assigned to it :

if [ -e '/etc/default/grub' ]; then
  command sed -i -e "s/\(GRUB_CMDLINE_XEN_DEFAULT=.*\)\"/\1 dom0_max_vcpus=${DOM0_CPU_COUNT} dom0_vcpus_pin=1\"/" \
         '/etc/default/grub'
fi
if [ -e '/boot/grub/menu.lst' ]; then
  command sed -i -e "s/\(xenhopt=.*\)/\1 dom0_max_vcpus=${DOM0_CPU_COUNT} dom0_vcpus_pin=1/" \
         '/boot/grub/menu.lst'
fi

Update the Xen kernels configuration:

command update-grub

Load the new configuration:

command reboot

Network configuration

Disable the current network settings :

command sed -i -e 's/^(network-script .*).*$/# \0/' \
            -e 's/^(vif-script .*).*$/# \0/' \
         "/etc/xen/xend-config.sxp"

Apply the chosen network settings :

command sed -i -e "s/^#[ ]*\\((network-script.*network-${XEN_NETWORK_MODE}).*\\)\$/\\1/" \
            -e "s/^#[ ]*\\((vif-script.*vif-${XEN_NETWORK_MODE}).*\\)\$/\\1/" \
         "/etc/xen/xend-config.sxp"

Make sur the hotplugpath.sh file exists :

if [ ! -e '/etc/xen/scripts/hotplugpath.sh' ]; then
  command touch '/etc/xen/scripts/hotplugpath.sh'
fi

Reload Xen settings :

command test -x '/etc/init.d/xen' && /etc/init.d/xen restart
command test -x '/etc/init.d/xend' && /etc/init.d/xend restart

Preventing known problems

Memory squeeze in netback driver

After some time, this error can appear in dom0 log :

xen_net: Memory squeeze in netback driver.

Limit the memory used by the dom0 (which quantity is set in this how-tos parameters):

if [ -e '/etc/default/grub' ]; then
  command sed -i -e "s/\(GRUB_CMDLINE_XEN_DEFAULT=.*\)\"/\1 dom0_mem=${DOM0_RAM}M\"/" \
         '/etc/default/grub'
fi
if [ -e '/boot/grub/menu.lst' ]; then
  command sed -i -e "s/\(xenhopt=.*\)/\1 dom0_mem=${DOM0_RAM}M/" \
         '/boot/grub/menu.lst'
fi

Apply the setting:

command update-grub

Prevent dom0 memory to lower under this limit:

command sed -i -e "s/^[# ]*\((dom0-min-mem\).*\().*\)$/\1 ${DOM0_RAM}\2/" \
         '/etc/xen/xend-config.sxp'

Restart the system to load the settings:

command reboot

/dev/mem: mmap: Bad address in domU

This error source is dmidecode. dmidecode try to access to lowlevel hardware informations that are not available when using Xen. This error can be ignored.

Thanks

• Thanks to Xen (en) developers.
• Thanks to Falko (en) for The Perfect Xen 3.1.0 Setup For Debian Etch (en).
• Thanks to  Xen Networking (en) authors on Xen wiki (en).
• Thanks to Crucial Paradigm (en) for Xen Bug: xen_net: Memory squeeze in netback driver (32bit PAE) (en).
• Thanks to BitFolk for Customer informations (en).