linux-vserver_administrators_guide (2003.12.09 16:00:00)

---

Warning: This is a work in progress. It will take some time before it will be finished.

The contents of this document isn't checked by other people. {hmm, is that so? ;}

(Thanks Herbert!)

Don't use this documentation to configure a production system.

Credits

This document is based on other linux-vserver documentation and mailinglist posting.

Some documentation regarding to the configuration of vservers is taken from the vskel documentation.

Preface

In this document we will try to provide system administrators a guide for installing, configuring and maintaining virtual servers using the linux-vserver project.

In this guide we will try to cover the differcences between the linux distributions.

A. Introduction

The linux-vserver project lets you run virtual linux servers on top of a linux machine. It uses a patch kernel and an tools package.

B. Installation

1. Requirements

• Supported platforms

• alpha

• i386 and higher (and compatible)

• ia32 / ia64

• mips / mips64

• parisc (HP PA-RISC)

• ppc / ppc64

• sparc / sparc64

• x86_64 (AMD64)

• uml

• Supported distributions

  Linux-VServer should run on most popular distributions.

• System requirements

  Due to the very low overhead of Linux-VServer, it doesn't have special system requirements. If you computer can run a standard linux distribution, it should be able to run linux-vserver. The overhead of the additional vservers depends on the services you run inside these vservers.

2. Download

• Kernel patch

  You can find the latest kernel patch at the linux-vserver site. Make sure you download a patch for the kernel version you use.

  You can download the linux kernel at http://www.kernel.org

  (please use a mirror)

• Tools

  You can choose between two tool packages. Vserver tools is the 'original' package, util-vserver is a new implementation, which follows the kernel development more closely.

  • [vserver]
  • [util-vserver]

• Distribution specific packages
  • Redhat
    • [vserver]
  • Mandrake
    • [vserver]
  • Debian
    • [vserver]

3. Patching the kernel

After downloading the linux kernel and the vserver kernel patch, you have can apply the patch to the kernel source and build you own kernel.

3.1. Standard kernel compile

• cd /usr/src
• tar -xzf linux-2.4.XX.tar.gz
• copy the patch to /usr/src
• cd linux-2.4.XX
• patch -p1 <../patch-2.4.XX-vsYYY.diff
• make menuconfig (or make oldconfig, or make xconfig)
• make bzImage
• copy the kernel image from .../linux/arch/i386/boot/bzImage to the location where your regular bootable kernel is found

  (this can be /vmlinux or the /boot/ directory)

• make modules_install
• update your boot manager (lilo or grub, or ...)

3.2. Debian

You can install a kernel as described in previous section, our you can use some of the debian tools. Debian contains a tool, kernel-package, which lets you build a kernel an put it in a debian package.

In case you use the kernel source included in the debian distribution, you can download a linux-vserver patch for it [here]

• cd /usr/src
• tar -xzf linux-2.4.XX.tar.gz (or tar -xjf in case it's a .bz2 archive)
• copy the patch to /usr/src
• cd linux-2.4.XX
• patch -p1 <../patch-2.4.XX-vsYYY.diff
• make menuconfig (or make config, or make xconfig)
• make-kpkg --revision=NAME.1.0 kernel_image
• dpkg -i kernel-image-NAME.1.0.deb

In case you have many servers, you should look at the kernellab tool to manage you kernel configurations.

3.3. Redhat

3.4.

C. Configuration host server

1. Configuring vserver

After installing a customized kernel and the vserver or util-vserver tools, you can find the configuration options in the file:

/etc/vservers.conf

configuration options:

• BACKGROUND= yes | no

  Start the vservers on tty9, in background so the rest of the boot process ends early.

• VSERVERS_ROOT= <path to vservers> (only in vserver package)

  This sets the location of the vservers. In case it's empty, the default value is /vservers. The util-vserver package doesn't use this option. Its location of the vservers is set during the installation of util-vserver.

2. Daemons on the host server

The daemons running on the host server will by default bind to all interfaces on the machine, including the alias interfaces used by the vservers.

This will result in conflicts between the daemons running on the host system and the daemons running in the vservers. To solve this, we have to bind the daemons running on the host server to the IP-number of the host server. This can be done by using the wrapper scripts provided by the vserver tools or util-vserver packages. A alternative can be changing the configuration of the daemons. Some daemons (ssh, xinetd) have a configuration option to bind the on one interfaces.

2.1. Startup wrapper scripts

On systems using system V startup scripts (like redhat), you can use wrapper scripts to bind the daemons to one interface. Replace inetd with xinetd.

• v_gated.sysv
• v_httpd.sysv
• v_named.sysv
• v_portmap.sysv
• v_sendmail.sysv
• v_smb.sysv
• v_sshd.sysv
• v_xinetd.sysv

2.2. Configuring daemons

Use this option in case the startup wrapper scripts don't work on your distribution, or you don't want to change the startup scripts of your daemons.

• SSH

  config file: sshd_config

  ListenAddress? 192.168.1.20

• inetd/xinetd

  You can't bind inetd to a interface, replace it with xinetd.

  config file: xinetd.conf

  defaults

  {

  bind = 192.168.1.20

  }

D. Virtual hosts

1. Installation

The script newvserver is used to create a new vserver. Some distribution specific packages include customized version of this tool (we know the debian package of vserver does include a customised version).

Run newvserver -h or newvserver --help to get the help.

2. Configuration

2.1. Configuration file

Each virtual server has one configuration file controlling its behavior. The configuration is located in /etc/vservers and has the name of the vserver followed by the extension .conf.

This is a pseudo shell script where variable are setup using shell assignment (VAR=value). The script is simply sourced by the vserver(8) utility. Here are the various variables you can set:

You don't have to set all of these options (only hostname, domain and IP-number are required)

• S_CONTEXT=

  Sets the vserver's security context number. If not specified, a free context is allocated dynamically. You usually only need to use fixed context numbers if you are using context quotas/disk limits, but as it doesn't hurt, it's recommended.

• IPROOT=

  Select the IP address to be assigned to this virtual server. Entering or starting the vserver will enable this IP as an interface alias if it does not already exist as an interface on the host server, and stopping the vserver will disable the alias.

  Up to 16 IP numbers may be assigned. Separate each IP address with a space.

The syntax is: [device[.vlan]:]IP[/netmask]

The netmask may be specified in quad notation (255.255.255.0) or by telling the number of bits assigned to the network part (A class C network is a /24 network with 24 bits assigned to identify the network and 8 (32-24) bits assigned to identify the host.

If the device is omitted, the IPROOTDEV devices is used. If the netmask is omitted, the device netmask is used.

Examples:

IPROOT="1.2.3.4"

IPROOT="1.2.3.4 2.3.4.5"

IPROOT="eth0:1.2.3.4 eth1:2.3.4.5"

IPROOT="eth0:10.1.2.3/255.255.255.0 eth0:10.1.3.12/255.255.255.128"

(see http://vserver.13thfloor.at/Stuff/VServer-IP-Setup-0.1.txt for details, e.g. how to not create an alias at all)

• IPROOTMASK=""

  Netmask for the IPROOT. If not specified, the netmask is calculated from IPROOTDEV.

• IPROOTBCAST=""

  Broadcast address for the IPROOT. If not specified, the broadcast is calculated from IPROOTDEV.

• IPROOTDEV=

  Select which network interface the IP alias will be created on (when not overridden in IPROOT).

  IPROOTDEV="eth0"

• S_HOSTNAME=

  Set the hostname (FQDN) of the vserver. If empty, the hostname of the host server is used.

• S_DOMAINNAME=""

  Set the NIS domain for the vserver. If empty, the domain of the host server is used. Set to "none" to have no NIS domain set.

• S_NICE=""

  Set the priority level (nice) of all processes in the vserver (from -20 [highest] to 19 [lowest]). Even root will not be able to raise it. See nice(1).

• S_FLAGS=

  Sets various flags for the vserver's security context. Separate each flag with a space.

  1) lock: Prevent the vserver from setting new context

  2) sched: Merge scheduler priority of all processes in the vserver so that it acts like a single one

  3) nproc: Limit the number of processes in the vserver according to ulimit (instead of a per-user limit, it becomes a per-vserver limit)

  4) private: No other process can join this security context, even root

  Example:

  S_FLAGS="lock nproc"

• ULIMIT

  You can set various ulimit flags and they will be inherited by the vserver. Enter the command line arguments you would give to the ulimit command.

  Example:

  1) ULIMIT="-HS -u 200"

  Combined with the nproc S_FLAGS, this example would limit the vserver to a maximum of 200 processes

  ULIMIT=""

• ONBOOT

  If yes, this vserver will be started at boot time. Set to no if you do not want it to start automatically.

  ONBOOT="yes"

• PRIORITY

  This control the start and stop order of the virtual servers (by the vservers service). A lower value means the vserver is started earlier. This is useful when some virtual server depend on other to run. For example you may have a virtual server providing LDAP services to the other. Failing to start this one first creates numerous problems.

• S_CAPS

  Allows you to set the vserver root user's capabilities. By default, vservers run with a limited set so that root can run in a vserver without worrying about him taking over the machine. Where necessary you may give a little extra capabilities. Separate each capability with spaces.

  You can find more info on the use of capabilities in linux, in the linux source tree. Look in the file include/linux/capability.h

  (for most users, this should be in /usr/src/linux/include/linux/capability.h)

The next capabilities are enabled by default.

CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, CAP_FSETID, CAP_KILL, CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICE, CAP_SYS_CHROOT, CAP_SYS_PTRACE, CAP_SYS_TTY_CONFIG, CAP_LEASE

You can extend the list of capabilities with the next capabilities. This does make the host system less secure. Use these options only if you need them for the configuration of a vserver.

• CAP_SETPCAP

transfering permitted capabilities

• CAP_LINUX_IMMUTABLE

modifying immutable and iunlink flags

• CAP_NET_BROADCAST

allow network broadcasting/multicasting

• CAP_NET_ADMIN

network interface configuration setting promiscuous mode (sniffing), multicasting and routing tables

• CAP_NET_RAW

use of raw and packet sockets

• CAP_IPC_LOCK, CAP_IPC_OWNER

ipc owner check and mlock

• CAP_SYS_MODULE

insert and removal of kernel modules

• CAP_SYS_RAWIO

ioperm/iopl access and usb messages

• CAP_SYS_PACCT

process accounting

• CAP_SYS_ADMIN

this list would be too long, it basically alows to do everything else, not mentioned in another capability.

• CAP_SYS_BOOT

allow reboot/halt

• CAP_SYS_NICE

allow raising priorities, and scheduler stuff

• CAP_SYS_RESOURCE

override resource limits, override quota and reserved space modify data journaling, consoles, keymap

• CAP_SYS_TIME

manipulation of the system clock

• CAP_MKNOD

creation of kernel nodes (Required to run a systemimager-server in a vserver.)

Example:

S_CAPS="CAP_NET_RAW" # allows raw sockets. Let the vserver send raw packet over the network (Enable ping and traceroute for one)

S_CAPS="CAP_SYS_RESOURCE" # This allows a application inside a vserver to enlarge its ulimit values. You need this to run bind9.

S_CAPS="CAP_SYS_ADMIN" # Let the vserver do many things, including mounting devices and network shares.

S_CAPS=""

2.2. Startup scripts

To run a command on the host server at startop or shutdown of a vserver, you can put a script in /etc/vservers/.

You have to give it the same name as your vserver and .sh as extention. (call it vserver1.sh for you server1)

example:

This script lets you map a directory on a other disk in you vserver (you can use it to share a directory between virtual servers.

(you have to put a #!/bin/sh on the first line of the script)

case $1 in

pre-start)

mount --bind /home2/home /vservers/server1/home

;;

post-start)

;;

pre-stop)

;;

post-stop)

umount /vservers/server1/home

;;

esac

E. Administation of linux-vserver

1. The tools

• vserver

  This command lets you manage the vservers.

  usage: vserver VSERVER <command>

  The next commands are supported by vserver

  • build : Create a virtual server by copying the packages of the root server
  • enter : Enter in the virtual server context and starts a shell. Same as \"vserver name exec /bin/sh\"
  • exec : Exec a command in the virtual server context
  • suexec : Exec a command in the virtual server context uid
  • service : Control a service inside a vserver.

    vserver name service service-name

    start/stop/restart/status

  • start : Starts the various services in the vserver, runlevel 3
  • stop : Ends all services and kills the remaining processes
  • running : Tells if a virtual server is running. It returns proper exit code, so you can use it as a test.
  • status : Tells some information about a vserver.
  • chkconfig : It turns a server on or off in a vserver.
  • --nodev : Do not configure the IP aliases of the vserver. Useful to enter a vserver without enabling its network and avoiding conflicts with another copy of this vserver running elsewhere.
  • --silent : No informative messages about vserver context and IP numbers. Useful when you want to redirect the output

• newvserver

  This script installs a new vserver.

• vpstree

  Presents the processes running in all virtual servers

• vtop

  A version of top which shows all running processes in all virtual servers and on the host server. Look at "man top" for the options.

• vkill

  Lets you kill a process running in a vserver. It kills the process inside the vservers this running in.

• vps

  The vps utility is simply a wrapper for the ps program. It shows all processes running on the computer (on host and in the virtual servers). It does have the same syntax as the command ps.

• vserver-stat

  This utility shows informations about all the running vservers on the machine.

• vrpm

  This is simply a wrapper for the rpm utility. It enables rpm to manage a specific, several or all virtual server(s) with one command. vrpm is executed in the root server.

  Using the unify option of vrpm, you can save diskspace by share files between a template vserver and you other vservers. It replaces the files by symbolic links.

2. Configuration files and startup scripts.

• /etc/vservers.conf

• /etc/init.d/vservers

• /etc/init.d/rebootmgr

• /etc/vservers/VSERVER.conf

• /etc/vservers/VSERVER.sh

3. Daemons

• rebootmgr

F. Configuration tips

security

• do not enable features you do not need
• do not give linux capabilities to vservers without really good reason
• do not run services on the host, except for the minimal set (sshd, ntpd¹, syslog)
• make sure that the 000 barrier is there and working
• do not use local loopback inside a vserver
• make sure your /proc entries are restricted
• try to follow security fixes (kernel)

stability

• do not enable SMP in kernel used on single processor machines (unless Xeon/P4 HT)
• do not select kernel features/drivers your hardware doesn't need
• avoid preemption
• do not run services on the host, except for the minimal set (sshd, ntpd, syslog)
• avoid cronjobs starting on all servers at once (smart daily cron rotation)
• do not use experimental or development kernels/patches unless you absolutely need the provided features

¹ - Contrary to what the FAQ says, it *is* possible to run ntpd within a vserver. See VirtualiseNTP.

G. References

• Linux-Vserver FAQ

---

Document last modified Thu, 16 Sep 2004 00:02:04

Search: