alpha util-vserver (2005.12.01 05:02:01)

---

The [alpha branch] of util-vserver

RELATED PAGES

• Namespaces - Interesting information regarding Namespaces and alpha util-vserver
• [The Great Flower Page] - Detailed Information on VServer configuration
• InitStyles for vservers using alpha util-vserver

INTRODUCTION

The naming scheme of the tools is either x.y or x.y.z. When

• z is missing, it is a stable release
• z is small (<90), it is a development release of the stable branch
• z is >=90 but <190, it is a release candidate of the stable branch
• z is >=190, it is an experimental snapshot of the alpha branch

Please note that the paths used in this documented are assuming an installation with packagemanagement. When using the plain ./configure, the path /usr translates to /usr/local, /etc to /usr/local/etc, and /var to /usr/local/var.

A description of the new configuration can be found on the [Flower Page]. But note that usually you don't create this configuration manually but use the tools as presented below.

ATTENTION

Beginning with 0.28.200, util-vserver will use CLONE_NS and RBIND to mount directories within vservers. Because some kernel support is missing at time of writing (Feb 10 2004), the mounted directories are not visible when entering the vserver with vserver <vserver-id> enter. Stopping them (vserver <vserver-id> stop) can have some issues too.

When you do not want to test this new feature, it is strongly recommended to do

# touch /etc/vservers/<vserver-id>/nonamespace

for every vserver. This flag can be set globally in /etc/vservers/.defaults too.

Since 0.29.197 it is possible to enter the vserver with

# vnamespace --enter <xid> vserver <vserver-id> enter

where <xid> is the xid of the running vserver. The nonamespace flag must not be set in this case.

Building the util-vserver tools

The util-vserver tarball contains a reference .spec file which can be used to build binary RPMs on Fedora Core 2 systems with the

$ rpmbuild -ta util-vserver-<version>.tar.bz2

command. On systems without the xalan XSL processor or without a recent dietlibc, the options --without xalan or --without dietlibc can be appended to the command above.

When prefering manual builds, you can execute the usual

$ ./configure && make && make install

sequence. Please note, that without an explicit --prefix option, everything will be installed and assumed under /usr/local. E.g. the configuration will not be at /etc/vservers, but at /usr/local/etc/vserver.

To enable the "standard" paths, read the ./configure --help output and set the responsible paths manually.

Vserver creation

This has only been done within a Fedora Core 1 host yet, but other distros should work.

Possible/tested vservers are:

• Red Hat Linux >=9 and Fedora Core 1 vservers; the appropriate command is

# vserver <vserver-id> build -m apt-rpm <other-options>* -- -d fc1 # for FC1 vserver, use '-d rh9' for RHL 9

before doing this, you need a recent apt (e.g. from [fedora.us]) and have to configure your nearest mirror in /etc/vservers/.distributions/<dist>/apt/sources.list.

With applied quota-patch it is strongly recommended to use the --context <ctx> option.

• Fedora Core 3 vservers + hosts; the command is

# vserver <vserver-id> build -m yum <other-options>* -- -d fc3

this requires a recent enough 'yum' (>= 2.1.12 should be ok). BEWARE: this method has not been tested very much and it can not be guaranted that the underlying 'vyum' does not read files from the guest.

• generic RPM based guests plus a recent rpm on the host; the command is

# vserver <vserver-id> build -m rpm <other-options>* -- -b <base-directory> <pkg-list>*

which installs a list of rpm-packages without resolving dependencies. <pkg-list> is a file with one package-name per line which will be searched below <base-directory>. Alternatively, a preconfigured packageset can be specified with -d <distribution> (currently, only fc3 is the only supported distribution).

• Debian woody and sarge vservers; the command would be

# vserver <vserver-id> build -m debootstrap <other-options>* -- -d sarge # or woody

you can configure the mirror with '-m <mirror>' or by putting it into the file /etc/vservers/.defaults/apps/debootstrap/mirror. This command assumes either an installed debootstrap package, or will download it from the URI configured in /etc/vservers/.defaults/apps/debootstrap/uri (see /usr/lib/util-vserver/defaults/debootstrap.uri for the default value). You can setup a local path too by beginning it with '/'.

• a basic skeleton (configuration entries, /dev entries); the command would be

# vserver <vserver-id> build -m skeleton

The first two options need a good internet connection; it is possible that more than 100MB will be downloaded. Alternatively and when you are experienced with the apt-tools, you can create your own mirror which is based on the CD-ROMs.

<other-options> can be e.g. --hostname or --interface; for complete list see vserver - build --help output

Examples

• # vserver foo build -m apt-rpm --hostname vs.foo.org --netdev eth0 --interface 192.168.3.1/21 --context 42 -- -d fc1

Creates the vserver foo with the Fedora Core 1 distribution and apt-rpm method. Its hostname will be vs.foo.org and the name shown by tools like vps or vserver-stat be bar. The static context will be 42 and the assigned interface eth0 with ip 192.168.3.1 and prefix 21 (== mask 255.255.248.0).

• # vserver foo build -m rpm --hostname vs.foo.org --netdev eth0 --interface 192.168.3.1/21 --context 42 -- -d fc3 -b /media/cdrom/Fedora/RPMS

Creates the vserver foo like above but with a predefined packageset. These packages will be searched on a mounted cdrom.

• # vserver foo build -m debootstrap --hostname vs.foo.org --netdev eth0 --interface 192.168.3.1/21 --context 42 -- -d sarge -m ftp://ftp.debian.org/debian

Creates the vserver foo with the debian sarge distribution using a network installation from the server ftp.debian.org.

Directory/vserver unification

The alpha branch of util-vserver reimplemented vunify so that it can unify arbitrary directories without requiring information from package-management (rpm). It introduces the concept of reference-vservers too. It has rsync-like excludelists, so that you can e.g. exclude anything under /etc/ except /etc/termcap; the corresponding excludelist would be

+/etc/termcap

/etc

Unifying directories

just type

# /usr/lib/util-vserver/vunify --manually <dst-path> <dst-excludelist> <reference-path> <reference-excludelist>

It is possible too, to specify multiple reference-paths; when a file is unificable, it will be linked with the file of the first matching reference-path.

Unifying vservers

to do this, the vunify application must be configured by

# mkdir -p /etc/vservers/<refserver>/apps/vunify

# mkdir -p /etc/vservers/<vserver-id>/apps/vunify

# ln -s /etc/vservers/<refserver> /etc/vservers/<vserver-id>/apps/vunify/refserver.00

This will make <refserver> a reference vserver of <vserver-id>. You can create multiple refservers by creating more "refserver.X" symlinks. These links will be handled in alphanumerical order. Files in these vunify directories which are named "exclude" will be assumed as static excludelists. When packagemanagement is configured for the vserver, configuration-files will be added to the excludelist.

When typing

# /usr/lib/util-vserver/vunify <vserver-id>, or

# vserver <vserver-id> unify

the unification happens. Details and debuginformation can be turned on by specifying one or more -v options.

vhashify

vhashify is the successor of vunify but does not require a reference vserver. Instead of, it calculates unique hash values for files, creates reference files on a certain place in the filesystem and unifies against these files. To vhashify a vserver:

• When not already done by your packagemanagement, create a global hash database-directory:

# mkdir -p /etc/vservers/.defaults/apps/vunify/hash

• In each used filesystem which contains unifyable files, create a hash directory and link it into the database-directory:

# mkdir -p /vservers/.hash

# ln -s /vservers/.hash /etc/vservers/.defaults/apps/vunify/hash/0

• Prepare the vserver for unification by creating the vunify application directory (vhashify shares some parts of its configuration with vunify):

# mkdir -p /etc/vservers/<vserver-id>/apps/vunify

Static excludelists can be configured in the way like above also

Now, you can hashify the vserver with

# vserver <vserver-id> hashify

/usr/lib/util-vserver/vhashify allows to hashify single directories also; the --help output will tell you, how.

NOTE for non-rpm based linux distro's

You need to specify what files to exclude from the unify because vhashify doesn't recognize portage or anything other than rpm based package managers.

Example command:

1. vhashify --manually <-nv> --destination /etc/vservers/.defaults/apps/vunify/hash/ /vservers/<vserver-id> exclude <path to exculde list>

1. !Note everything that isn't excluded will be unified so please run with the -nv switch to test first

The initscript

Since 0.29.202, util-vserver has a vservers-default initscripts. This script is calling the start-vservers program which:

• starts and stops vservers in parallel; since lot of startup/shutdown time is spent in waiting for something, the load should be not affected by this but the overall time will be reduced significantly

• understands a -m <mark> option. Most vservers can be started very late (e.g. customer vserver) but there may be vservers like DNS or LDAP servers which must be started early in the bootprocess. To make this -m option possible, a vserver can get a tag in

/etc/vservers/<vserver-id>/apps/init/mark

Content of this file is a string; the shipped vservers-default initscript handles all vservers having default as this string. The initscript can be easily modified to support other tags, for details just look into the script.

The vshelper userspace reboot helper (since 0.29.206)

The userspace helper is configured through the .../apps/vshelper directories. By placing a disabled file there, it can be disabled either globally (when using /etc/vservers/.defaults), or for a single vserver only. The default actions are configured to restart the vserver on 'restart' and to stop it at 'halt' or 'poweroff' events.

To make vshelper working, it was required to enable the CAP_SYS_BOOT capability for all vservers. By removing this capability explicitly, this could be changed. But since the vshelper is used for synchronisation of minit-style vservers also, touching of the disabled file is the preferred way.

Examples

• disable vshelper for the entire system

# touch /etc/vservers/.defaults/apps/vshelper/disabled

• disable vshelper for vserver foo only

# touch /etc/vservers/foo/apps/vshelper/disabled

• notify administrator when vserver bar dies

# echo 'notify' >/etc/vservers/bar/apps/vshelper/action

# cp my-notify-script /etc/vservers/.defaults/apps/vshelper-methods/notify

(my-notify-script is a script expecting xid and event as parameters; the $VSERVER environment variable with the id of the vserver is set)

• prepare a minit based vserver

# vserver <id> enter

# ln -s /sbin/hard-reboot /etc/minit/ctrlaltdel/run

# echo 'RESTART' >/etc/minit/ctrlaltdel/params

# touch /etc/minit/ctrlaltdel/sync

Kernel 2.4 to kernel 2.6 transition

Since the barrier and immulink implementation differs between kernel 2.4 and kernel 2.6, the following problems will occur when under kernel 2.4 created vservers will be used under kernel 2.6:

• the chmod 000 barrier does not work anymore; there should be no problem when Namespaces are enabled (the default), but without Namespaces the known chroot(2) exploits will work. The barrier can be checked with

# showattr -d /vservers

Things are fine when you see an uppercase 'B' like in

---Bui- /vservers

Else you should execute

# setattr --barrier /vservers

• the 'immulink' attribute (e.g. set by 'vunify') from kernel 2.4 is a simple 'immutable' attribute under kernel 2.6. This means that files can be neither modified nor deleted; therefore, package-mananagement will not work anymore. A quick hack (please read the full text before executing the commands) would be

# lsattr -R 2>/dev/null /vservers | grep -- '----i------t-' | cut -d ' ' -f 2 | xargs setattr --iunlink

# lsattr -R 2>/dev/null /vservers | grep -- '----i------t-' | cut -d ' ' -f 2 | xargs chattr -t

(using only the reference vservers instead of /vservers would be probably more efficiently)

Obviously, these commands are not very reliable:

• all affected vservers must be stopped; else symlink attacks are possible (note by pflanze: actually, to be safe from symlinks pointing outside of /vserver/* subtrees, one should use something like find -type f -print0 |xargs -0 lsattr | ... instead (untested).)

• filenames with whitespace are not accepted

Therefore, you should verify with

# lsattr -R 2>/dev/null /vservers | grep -- '----i------t-'

that all files have been updated.

---

Document last modified Thu, 05 Jan 2006 21:56:30

Search: