Integrating Xen2 With Mandriva 2005 LE

The following is applicable to Xen 2.0.6.

1. Do a minimal install of Mandriva 2005 LE + the following packages for the sake of convenience:
* wget
* screen
* openssh-server
* openssh-clients
* slocate
* lynx
* rsync
* traceroute
* vim-common
* vim-enhanced
* drakwizard
* drakxtools
* ncftp
* lsof

If you intend to use LVM backed VBDs (Virtual Block Devices) for domU, install the lvm2 package too.

2. Go to http://urpmi-addmedia.org and configure main, contribs, and updates urpmi media sources.

3. Download the stable source tarball from the Xen Project download page here.

4. urpmi the following packages and accept their dependents when prompted:

* python-twisted
* bridge-utils
* gcc
* gcc-c++
* ncurses-devel
* curl-devel
* zlib-devel
* python-devel
* glibc-devel

5. Untar the tarball and run the install.sh script. That tests for dependencies. There should not be any after having installed the packages in the previous step but if there are, do:

urpmf fileName

where fileName is the thing that is missing.

6. cd into the xen-2.0 directory and run "make dist". Again, if you have problems with missing packages, install them as above.

This should fetch the pristine kernel sources from kernel.org, apply the Xen patches to it, and build the new kernel. In all likelihood, the default configuration will not be suitable as there will be either too many or too few options that were selected. I found that it did not build anyway.

7. cd into the linux-2.6.11-xen0/ directory and run:

make ARCH=xen menuconfig

to run the kernel config tool. The .config file that worked for me, which is appropriate for the specific machine I built it on but not necessarily for any other, can be found here.

Note: I had to repeat the menuconfig/make dist cycle several times before I got a clean build. I got errors related to NFS and USB modules so I deselected them, ran make dist, and once I got a clean build, I selected those options again and it worked.

8. Once you have a clean build, run install.sh to install the kernel and modules to their proper places. Do not remove the Mandriva kernel or its modules. You may need it if you run into trouble later.

10. mv /lib/tls /lib/tls.disabled

11. Edit /boot/grub/menu.lst and add:

title Xen2 - 2.6.11 kernel
kernel /xen-2.0.gz dom0_mem=65536 acpi=ht resume=/dev/sda6
module /vmlinuz-2.6-xen0 root=/dev/sda5 ro console=tty0

Do not remove the entries for the Mandriva kernel.

Notes:
a) The leading /boot is not used.

b) The docs state that specifying dom0_mem=64M should work. It did not when I did a source installation. The solution is to specify in kilobytes with no trailing "K" or "k".

12. WARNING: If the system is running at a higher level of msec, Shorewall would normally be started. Shorewall will fail to start resulting in no remote access. This would obviously be a problem if you have been doing all of this via ssh. Turn Shorewall off, at least for now by doing:

chkconfig shorewall off

Reboot the system. It should now be running the Xen kernel in dom0. If you see a message like the one between the hash marks flash by on the console before the machine reboots, reboot with the Mandriva kernel and ensure that dom0_mem is specified as an integer representing kilobytes without a "K" or "k" after it.

#################################################
(XEN) Initial guest OS requires too much space
(XEN) (8MB is greater than 0MB limit)
****************************************
Could not set up DOM0 guest OS
Aieee! CPU0 is toast...
****************************************
Reboot in five seconds...
#################################################

If you do not have a remote console, you not will not be able to see this message, or any others that are sent to the console, and unless you designated a fallback kernel in menu.lst, the machine will be stuck in an infinite loop of starting, failing, and restarting. All of this is very risky to do remotely so it would be highly advisable to have some mechanism for rebooting the remote machine remotely using a known good kernel, having remote hands, or a remote console, whether it be a serial console or a KVM over IP unit.

13. service xend start

It will not start because the install.sh script did not create some key directories. Look in /var/log/xend-debug.log to determine which directories are missing and create them like so:

mkdir -p /var/lib/xen/xend-db/domain/
mkdir -p /var/lib/xen/xend-db/vnet/xend-db/domain

A shell script that creates the above directories can be downloaded from here.

Try starting xend again. It should now start.

14. There are a few iptables related modules that Shorewall has to load that it does not by default for it to work with a Xen kernel so if you want to use Shorewall, add the following to /etc/shorewall/modules:

loadmodule iptable_mangle
loadmodule ipt_multiport
loadmodule ipt_conntrack
loadmodule ipt_TOS
loadmodule ipt_REJECT
loadmodule ipt_LOG
loadmodule ipt_state

A copy of /etc/shorewall/modules that worked for me can be found here.

15. service shorewall start

Note: This will prevent remote access via ssh. In order to enable remote access via ssh, start drakfirewall, let everything through, save, restart drakfirewall, and enable only those protocols that you want to allow through the firewall, e.g. ssh. That will enable remote ssh access.

16. Ensure that Shorewall, xend, and xendomains start by doing:

chkconfig ServiceName on

where ServiceName is xend, or whatever you want to start.