So you're eager to use Linux-VServer? Great! This document should help you to get there. It is assumed that you have basic knowledge about building a custom kernel, i.e. that you know which stuff to turn on in the kernel configuration. Of course some Linux-VServer-specific options are explained here (if I feel that there's a need for it).
Note: The instructions throughout this tutorial are doable with a regular user account, there's absolutely no need to do everything as root. If you need to become root, the instructions tell you about it!
You'll need the vanilla kernel sources (i.e. those from [kernel.org]) and (of course) a Linux-VServer patch for the kernel you intend to use. At the time of the writing of this document, the patches for the 2.6 kernel series are considered development, there's no stable release yet. Nevertheless, the patches tend to be at least as stable as the 2.6 kernel itself ;). The development patches can be found [here], more experimental patches (and sometimes also fixes which haven't made it into a release yet) can be found [here], as long as there is no stable release, it might prove useful to ask for the best choice of patches on our irc channel (#vserver @ OFTC). Alternatively You may also try one of the prebuild kernels for your distribution which can be found on the Homepage.
In this document, I'll just use Linux 22.214.171.124 and Linux-VServer 2.01. So now that we know what we need, let's get it.
First, we'll create a directory for our sources, if you already got one, feel free to skip this step and/or adjust the paths to your needs.
# Create a directory for our sources mkdir ~/src # Switch to that directory cd ~/src
Now that we have a place to store our sources, we need to fetch them. We start with the vanilla sources.
# Get Linux 126.96.36.199 sources wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-188.8.131.52.tar.bz2 # Extract them tar xjf linux-184.108.40.206.tar.bz2
Now it's time to get the Linux-VServer patch and apply it to the sources. While we're at it, I'll tell you a nice trick I learned from Bertl, that allows you to keep a lot source trees on your disk without using up lots of disk space (and this also speeds up 'diff' a lot, which is really nice if you do kernel-hacking). What we do is creating a hard-linked copy of our sources and patch this copy with the Linux-VServer patch. That way, only the patched files use additional disk space (and because hard-linked files are equal by definition, diff doesn't need to compare them ;).
So let's go.
# Get the Linux-VServer 2.01 patch wget http://www.13thfloor.at/vserver/s_rel26/v2.01/patch-220.127.116.11-vs2.01.diff.bz2 # Create a hard-linked copy of the vanilla sources, this will get the Linux-VServer patch applied cp -la linux-18.104.22.168 linux-22.214.171.124-vs2.01 # Switch to that new directory cd linux-126.96.36.199-vs2.01 # Patch the sources bzip2 -dc ../patch-188.8.131.52-vs2.01.diff.bz2 | patch -p1
Now we got two sources, the vanilla sources for 184.108.40.206 and the Linux-VServer sources for 220.127.116.11-vs2.01. You might ask "Why do we need two source trees at all? I only want one kernel!" and that's a good question. Here's one answer: Updates! If a new vanilla kernel is released, you can just download the patch from your version to the new version, if you had patched your one and only vanilla source tree, you couldn't do this. And if a new Linux-VServer patch is released, you can simply create another hardlinked copy of your vanilla sources and apply the new patch there. This can really save you time (and bandwith), because you can keep everything you might need, without wasting lots of disk space.
But be aware that this needs some disciple when hacking the source. Because hard-linked files share the same data on the disk, you need to make sure that your editor does The Right Thing, otherwise you might mess up all your source trees... (I might write some docs about working with hard-linked source trees sometimes... ;)
First, we'll create a hard-linked copy of our Linux-VServer-patched sources that we'll use to build our kernel, this way the original source tree is kept clean so that you can patch/diff/whatever it easily. I'll just use "-build" as a suffix for the directory name, you're of course free to call it the way you want it. Another reason for using another copy of the sources to build the kernel is that you can keep builds with different configurations or for different architectures, and remember it's not using any additional disk space :).
# Go back to our source directory cd ~/src # Create a hard-linked copy of the patched sources cp -la linux-18.104.22.168-vs2.01 linux-22.214.171.124-vs2.01-build # Switch to that new source tree cd linux-126.96.36.199-vs2.01-build
Now let's start configuring that copy, I'll only explain some of the Linux-VServer-specific kernel configuration options here (as of 2.0), the rest is up to you ;). You can choose whatever configuration method you like, for example:
# Configure the kernel using a ncurses based menu make menuconfig
Note: The kernel configuration allows you to select between different security implementations, one of them is the capability system. As you might know, Linux-VServer uses this system heavily, thus you just have to have this one enabled (it is possible to build it as a module, but for a Linux-VServer system, it's not worth the effort, as you need it anyway).
Now that we have configured the kernel the way we want it, it's time to build it.
# Build the kernel make
If you don't happen to have a really fast box, it is a good time to get a new cup of coffee now ;)
This really depends on your setup, if you don't know how to do this at all, please ask google for some howto or take a look at your distribution's documentation.
After you've installed your kernel and setup your bootloader, it's time to reboot. If the kernel doesn't boot, check your kernel and bootloader configuration, if you absolutely can't figure out what's wrong, try asking on irc (#vserver @ OFTC).
The kernel alone won't help you, you also need some tools to exploit all those new features you got, so let's get them.
The latest tools can be found [here]. For Linux-VServer on 2.6 kernels you should use the alpha tools, as only those support the new features (and don't worry, they're also pretty rock-stable). I'll use util-vserver 0.30.210 in this document, you should use whatever is latest.
As a first step, of course, we need to get the sources.
#if you are using debian or a debian drived distribution the add the source repository to you /etc/apt/sources.list #then do su apt-get build-dep util-vserver exit #the above will install many of the required tool to compile util-vserver # Go to our source directory cd ~/src # Get the sources for alpha util-vserver wget http://www.13thfloor.at/~ensc/util-vserver/files/alpha/util-vserver-0.30.210.tar.bz2 # Extract the sources tar xjf util-vserver-0.30.210.tar.bz2
Now that we got our sources, we need to do the usual steps: configure, make, make install.
While configuring the tools you may get some error messages about missing stuff, for example dietlibc, vconfig and e2fs headers. The error messages are accompanied by explanations what you should do, so read them carefully.
# Switch to the util-vserver source directory cd util-vserver-0.30.210 # Configure the sources (you may want to adjust settings here, the defaults work, but may not suite your needs) ./configure # Build the tools make # Become root su - # Install the tools make install # It's a good point to fix the /proc entries for the guests /etc/init.d/vprocunhide restart (this path depends on configuration, see output of 'vserver-info') # Back to our regular user exit
Now we got our tools in place and we're ready to build our first vserver!
Oh, wait, we're not ready yet! To be sure that your setup works fine, we got a test script that checks some basic functionality. So download the [testme.sh] script and check if everything's fine.
# Back to our source directory cd ~/src # Get the script wget http://vserver.13thfloor.at/Stuff/SCRIPT/testme.sh # Make it executable chmod +x testme.sh # Become root su # Run the test script ./testme.sh # Back to our regular user exit
If the script shows any errors, feel free to join us on irc (#vserver @ OFTC) or ask on the [mailing list]. If you ask on the mailing list, don't forget to include your kernel version, the version of the Linux-VServer patch, the version of util-vserver and of course the output of the test script.
This step is completely optional, but I guess some other people may want to do this too, so here we go.
On my system the default path where vservers are stored is /var/lib/vservers, but I'd like them in /home/vservers, since /home is the only partition where I have enough space.
To do that right it takes 2 things: tell the vserver tools about the new location and set a chroot barrier on that directory, so vserver guests can't escape (see below).
The first part is easy. /etc/vservers/.defaults/vdirbase is a symlink to the desired vserver base directory, we just need to change it. (It works this way if you have not enabled CONFIG_VSERVER_LEGACY in the kernel config. Otherwise you have to change the location in /etc/vservers/util-vserver-vars.)
(note: I doubt that this is the way to do it, because IIRC, the path is hardcoded into some scripts/tools ... specifying it at build time with --with-vrootdir=/home/vservers would be advised, and don't blame us for the /var/lib/vservers path, that's a debian oddity, all other distros we know of use /vservers -- Bertl)
The second step, setting the barrier flag on the base directory, is needed so guests can't escape from their chroot. On Linux 2.6 this isn't really necessary, since another mechanism is used to lock in the guests anyway, but it can't hurt to be on the save side.
# The old link pointed to /var/lib/vservers ls -la /etc/vservers/.defaults/vdirbase lrwxr-xr-x 1 root root 17 2005-06-18 13:26 /etc/vservers/.defaults/vdirbase -> /var/lib/vservers # So now we set it to our desired directory rm /etc/vservers/.defaults/vdirbase ln -s /home/vservers /etc/vservers/.defaults/vdirbase ls -la /etc/vservers/.defaults/vdirbase lrwxrwxrwx 1 root root 14 2005-06-18 23:26 /etc/vservers/.defaults/vdirbase -> /home/vservers # And set a chroot barrier flag on that directory, # using setattr from the vserver tools setattr --barrier /home/vservers/ # We use showattr to display the flags set on that dir, # -d is used to get the flags on the directory, rather then on the content. showattr -d /home/vservers/ ---Bui- /home/vservers/ # The uppercase B shows us that the barrier flag is set now
So, after you have (or have not) done this, we are ready to set up our first vserver.
OK; now we're ready for actually building our first vserver.
The building of the vserver is done with the vserver(8) command. The building process itself is documented in vserver-build(8) though. If you do not have this manpage installed on your system (the current [2005-06-19] util-vserver package in Debian Sid doesn't include it, for example), you can download the util-vserver source distribution, and view the manpage there directly (using man -l man/vserver-build.8).
There are quite a lot of methods to build a server, since I wanted a Debian guest, I chose debootstrap. The vserver-build manpage is really well written, so just look it up there if you want another method or don't really understand the example.
# We build a guest named "DebianSid", and use the debootstrap method to set it up. # Everything after the first -- is an argument for the debootstrap method, not for build itself. # So we tell it that we want the sid distribution, and it should use ftp.at.debian.org # as mirror, instead of the default one. # Everything after the second -- is interpreted as an argument to debootstrap itself. # --resolve-deps is required to directly bootstrap testing and unstable those days. vserver DebianSid build -m debootstrap -- -d sid -m ftp://ftp.at.debian.org/debian/ -- --resolve-deps # Watch debootstrap set up the guest.
This will setup a vserver in /home/vservers/DebianSid? (depends on the setting of vdirbase, cf. above). Management data is stored in /etc/vservers/DebianSid?. Additionally there will be some runtime data in /var/run/vservers/DebianSid? and /var/run/vservers.rev.
If something went wrong just delete these directories (except /var/run/vservers.rev, there just delete the entry that points to /etc/vservers/DebianSid?, if any), and re-run the vserver build command.
While you wait for debootstrap to finish it might be a good idea to read the [flower page], which is a comprehensive documentation of the configuration options for vserver and its guests. You may wish to save it on your harddisk and view it there, just to get rid of the annoying stylesheet.
So, debootstrap (or whatever method you used) is finished, you could start up our vserver, but it's not a good idea yet (of course I learned this the hard way ;). Most bootstrapping methods configure the guest as they would configure a real system. They install init scripts for all kinds stuff (setup of the console, setting the system clock, mounting drives, mounting /sys and /proc, ...). But you don't want the guests to do those things on startup/shutdown, since they are either handled by the host itself or by vserver. There doesn't happen anything fatal if you don't disable these scripts (since the guests just don't have the permission to do those things per default) but you'll get lots of scary error messages.
What you have to delete (and where the init scripts are stored in the first place) will depend heavily on your system. Below you can see what I did to clean up my init scripts.
cd /home/vservers/DebianSid/etc/rc0.d rm K20makedev K25hwclock.sh S30urandom S31umountnfs.sh S35networking S36ifupdown S40umountfs S90halt K89klogd cd /home/vservers/DebianSid/etc/rc6.d rm K20makedev K25hwclock.sh S30urandom S31umountnfs.sh S35networking S36ifupdown S40umountfs S90reboot K89klogd cd /home/vservers/DebianSid/etc/rcS.d rm S05keymap.sh S48console-screen.sh S50hwclock.sh S40networking S45mountnfs.sh S10checkroot.sh S02mountvirtfs rm S30procps.sh S35mountall.sh S36mountvirtfs S39ifupdown S30checkfs.sh S18ifupdown-clean S18hwclockfirst.sh cd /home/vservers/DebianSid/etc/rc2.d rm S20makedev S11klogd
Here's some hints on what you want to delete: everything that has something to do with mouting, networking (interfaces/ifupdown), the hardware clock (hwclock), console, creation of device nodes (makedev) and probably the halt/reboot stuff. You can also disable the klogd (kernel log daemon). It doesn't print error messages, but it doesn't get any data to log from the kernel, so it's pointless.
Stuff you probably don't want to kill includes sysklogd (the syslog daemon, not the kernel log daemon), cron, inetd (really depends on what you want to use the guest for), other logging stuff (bootlogd perhaps) and other stuff you might think you need.
When you're done pruning the init scripts of your guest you're finally ready to start it up. That's how it looks here:
vserver DebianSid start Starting system log daemon: syslogd. Starting internet superserver: inetd. Starting periodic command scheduler: cron. vserver DebianSid enter > echo In the vserver guest now! In the vserver guest now! > logout vserver DebianSid stop Stopping periodic command scheduler: cron. Stopping internet superserver: inetd. Stopping system log daemon: syslogd. Sending all processes the TERM signal...done. Sending all processes the KILL signal...done.
If it works like this everything is fine, and you can check out what you can do in the guest. You probably want to run
to get the Debian distribution installed to useable point. Please simply ignore the annoying error messagees about "LC stuff" - they vanish after base-config.
(note: using 'LC_ALL=C LANG=C' might avoid those 'issues')
In the next chapter we'll look at some configuration stuff. The rest of this chapter is dedicated for those that didn't get it to work that well yet.
I had some minor problems before getting the guest to start up so nicely, I hope this section helps others who run into the same problems.
Remember to bind the Services on the host with one IP only.
For Example the sshd on the host listen on _all_ IPs (includes all VServer)
Alternativ you can give the host another Port for ssh.
Try to run all services in a VServer, not on the host.
This one is not fatal, the guest starts up as normal anyway. It's annoying though.
vserver DebianSid start mount: wrong fs type, bad option, bad superblock on none, missing codepage or other error In some cases useful info is found in syslog - try dmesg | tail or so /etc/vservers/DebianSid/fstab:2:1: failed to mount fstab-entry
The last line tells us that the problem is not inside the guest, but it's in the vserver setup of the guest. The entry on line 2 in /etc/vservers/DebianSid?/fstab could not be mounted. The entry was to mount a tmpfs virtual file system on /tmp. Since I don't have tmpfs built into my kernel (CONFIG_TMPFS unset), this didn't work. Commenting out the line fixed the problem.
Also non-fatal, but annoying:
vserver DebianSid enter mesg: /dev/pts/1: Operation not permitted
I don't fully understand this one, but it seems to be a problem in util-vserver. The bug depends on the method how you got root in the host. If you got root by "su", the problem shows itself. If you used "su -" (which spawns a login shell as root), it won't.
(note: this is not a bug, this is the pts security in action, which prohibits the guest from messing with a pts allocated on the host -- Bertl)
So if this really annoys you so much, use "su -" to get root, or better yet, fix it in the vserver tools and send a patch :)
Yet to be written. This will be updated when I finally overcome my lazyness, or when someone else comes by and feels sorry for this page for not being updated in such a long time ;)
In the meantime, you can check out the [flower page] I mentioned above. Behind the angry fruit salad hides some valuable information.
Note: before you turn blind, try to see if your browser lets you switch the page style. On Firefox, you fid this option in the menu: View->Page Style Select 'boring' and it actually becomes readable.
This part covers the migration from the old (<= 1.2.10) config style to the new one (2.0+). It starts right away with the files necessary for making your old vservers work (this is based on a Debian guest system!). I used the [flower page] to understand the options as well as the skeleton setup method to create the minimum setup files.
If you followed this step-by-step guide, your directory holding the config files should be /etc/vservers/. If it's in a different place, replace /etc/vservers/ with the correct path corresponding to your setup!
Everything that doesn't end with a "/" is supposed to be a file and not a directory. The contents of a file is the value of the config option or in some cases a script.
none /proc proc defaults 0 0 none /dev/pts devpts gid=5,mode=620 0 0
<path-to-mounted-lvm> / none bind
These files are required in case you're using LVM, because the values can't be found automatically on vserver start.
For all the other options, check out the [flower page].