We currently migrate to MediaWiki from our old installation, but not all content has been migrated yet. Take a look at the Wiki Team page for instructions how to help or browse through our new wiki at wiki.linux-vserver.org to find the information already migrated.

17/04/05

Has got vserver running on a 6 way E4000. The procedure (in excessive detail) was:

Running Debian/SPARC 3.1 (sarge) ~ currently testing, installed:

kernel-source-2.4.27 version 2.4.27-8

kernel-patch-vserver version 1.9.5.3

then:

cd /usr/src

tar -xvj < kernel-source-2.4.27.tar.bz2

export PATCH_THE_KERNEL=yes

cd kernel-source-2.4.27

make-kpkg config menuconfig rootcmd fakeroot revision custom01 added-patches vserver --append-to-version +vserver --initrd binary_arch

(borrow the config file from 2.4.27-2-sparc64-smp)

cd ..

dpkg -i *.deb

reboot

All works fine.

22/04/05

... or so it seemed. A number of problems have arrisen.

Both the Debian 2.4.27 and 2.6.8 kernels build and run with the 1.9.5 patch however the exibit a number of problems:

After some analysis Bertl (via IRC) came to the conclusion that it was likely to be an interfacing problem between 32 bit userland apps and 64 bit kernel space. With some inspired hacking he managed to develop some syscall bindings that work for sparc64 and ppc, given they work when compiled as either 64 or 32 bit binaries this means they should work for sparc32 as well. He also says that the API used for the 2.6 kernels is mostly architecture and word size independant (except for a pointer used in passing filenames when dealing with quotas) and thus the chances of getting it working on 2.6 are much higher - that's where the effort should be (and is) going.

Tried 2.6.11.7-vs2.0-pre1 which seems to have the missing /proc/sys/vservers but is otherwise the same as 2.6.8.

It seems there are two possibilities (assuming that the kernel if functioning correctly - which we have (thus far) no reason to doubt):

1. compile up util-vserver as a 64 bit set of applications. This is really a hack rather than a long term solution as it would be cumbersome to distribute, etc.

2. integrate the new syscalls into util-vserver and it /should/ JFW (modulo the single use of pointers in the interface).

04/05/05

Bertl says that a 2.6.11 kernel with pre4, patched ( http://vserver.13thfloor.at/Stuff/DIETLIBC/ ) dietlibc 0.28 and vanilla

util-vserver 0.30.207 works.

15/05/05

Next attempt

kernel 2.6.11.9 patches, builds and runs fine with vs2.0-rc1

Building dietlibc requires some effort, the following patches

dietlibc-gethostbyname-fix01.diff

dietlibc-syscall-fix01.diff

from http://vserver.13thfloor.at/Stuff/DIETLIBC/ have to be applied and

.register %g2,#scratch

has to be added to libcompat/syscall.S and

  1. ifdef WANT_THREAD_SAFE

has to be commented out of dietfeatures.h

util-vserver-0.30.207 builds and links if -m64 is set in both CFLAGS and CXXFLAGS

chcontext and chbind both seem to work however vps and vtop don't as there seems to be a problem with unhiding the proc entries. Running vprocunhide gives:

Fixing /proc entries visibility.../usr/local/lib/util-vserver/vprocunhide: line 91: 11869 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11870 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11871 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11872 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11873 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11874 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11875 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11876 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11877 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11878 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11879 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11880 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11881 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11882 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11883 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11884 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11885 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11886 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11887 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11888 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11889 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11890 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11891 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11892 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11893 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11894 Bus error $_SETATTR -x "${params[@]}" "$@"

/usr/local/lib/util-vserver/vprocunhide: line 91: 11895 Bus error $_SETATTR -x "${params[@]}" "$@"

ERROR

A slightly more condensed example of this is:

  1. strace -o ~/trace -s 128 setattr --~hide /proc/uptime
umovestr: Input/output error

ptrace: umoven: Input/output error

  1. cat ~/trace
execve("/usr/local/sbin/setattr", ["setattr", "--~hide", "/proc/uptime"], [/* 13 vars */]) = 0

lstat(0xfffffecb, {...}) = 0

SYS_267(0, 0x3f, 0, 0, 0) = 65573

- SIGBUS (Bus error) @ 0 (0) -

+ killed by SIGBUS +

Which looks (to me) like an unaligned load / store but I could be wrong.

Attacking a copy of setattr with GDB it appears that vc_set_iattr is returning -1 (linked with SYS_267(0, 0x3f, 0, 0, 0) = 65573 I think), which is causing to perror to be called with errno = 90 (function not implemented I believe) and an un-aligned string. Somewhere in perror a bus error is occuring - although I can't figure out why as I can't replicate the bug in a test program.

New servers can be installed successfully but attempting to start them gives:

/usr/local/sbin/vserver: line 19: 28698 Bus error $_LOCKFILE "$1" $tmp $2

/proc/uptime can not be accessed. Usually, this is caused by

procfs-security. Please read the FAQ for more details

http://www.linux-vserver.org/index.php?page=Linux-Vserver+FAQ

16/05/05

Doener on #vserver suggests configuring with fscompat off, ala:

./configure --enable-apis=compat,v11,v13,net

and this seems to fix the problem :-D

vservers now install, vprocunhide seems to run, vps and vtop work (although vps gives ERR as the context of every process), but they don't appear to start properly:

  1. vserver XXXX start
/usr/local/sbin/vserver: line 19: 21726 Bus error $_LOCKFILE "$1" $tmp $2

/usr/local/sbin/vserver: line 92: 21746 Bus error $_LOCKFILE "$1" $tmp $2

save_ctxinfo: /usr/local/sbin/vserver: line 142: 21763 Bus error ${NICE_CMD[@]} $_CHBIND "${CHBIND_OPTS[@]}" $_EXEC_ULIMIT "$VSERVER_DIR/ulimits" $_CHCONTEXT_COMPAT "${CHCONTEXT_OPTS[@]}" "${CHCONTEXT_INIT_OPTS[@]}" $_SAVE_CTXINFO "$VSERVER_DIR" $_ENV -i -- $_CHAINECHO "${_IS_FAKEINIT:+$startsync_pipe}" "" $_CAPCHROOT "${CAPCHROOT_OPTS[@]}" . "${INITCMD_START[@]}"

An error occured while executing the vserver startup sequence; when

there are no other messages, it is very likely that the init-script

(/etc/init.d/rc 3) failed.

Common causes are:

method knows how to deal with this, but on existing installations,

appending 'true' to this file will help.

Failed to start vserver 'XXXX'

Looking into it, it seems the bus error is caused by one of the call to perror in lockfile. It seems there is some sort of alignment issue in dietlibc's perror. A compile of util-vserver with glibc gives different errors (vps now works fine, as does vtop, however vserver XXXX still fails - this time with only "wait(): No child processes").

Removing the lines:

if ( (unsigned int) errno < (unsigned int) __SYS_NERR )

message = sys_errlist [errno];

from lib/perror.c in dietlibc seems to remove the bus errors, am now left with:

lockfile: siginterrupt(): [unknown error]

lockfile: siginterrupt(): [unknown error]

save_ctxinfo: vc_get_task_xid(): [unknown error]

All of this is being cause by some strangness in the MACROS in util-vserver according to Doener` , notes that there is a bug in the configure script of util-vserver, the following is needed:

sed -i "s/enable_val/enableval/" configure

The bus errors in perror can be fixed by changing line sparc64/errlist.S to .align 8

With all of these changes and ./configure --disable-alternative-syscalls we now get the following result:

  1. vserver XXXX start
wait(): No child processes

which is the same as the glibc version. Progress.

17/05/05

Building all in 32 bit mode doesn't seem to help, same problems as before.

Bertl suggests replacing lib/syscall-alternative.h with http://vserver.13thfloor.at/Experimental/SYSCALL/syscall.h

util-vserver now appears to configure and run (32 bit mode) apart from starting vservers, fails with same error, ala

  1. vserver --debug XXXX start
+ shift

+ true

+ shift

+ break

+ OPTION_ALL=($OPTION_SILENT $OPTION_VERBOSE $OPTION_DEBUG $OPTION_DEFAULTTTY)

+ SELF=("$0" "${OPTION_ALL[@]}")

+ vserver=XXXX

+ cmd=start

+ test start '!=' build

+ allow_legacy=

+ VSERVER_DIR=/usr/local/etc/vservers/XXXX

+ allow_legacy=1

+ test -n 1

+ do_legacy=

+ test '!' -e /usr/local/etc/vservers/XXXX/legacy

+ test -d /usr/local/etc/vservers/XXXX -o '!' -e /usr/local/etc/vservers/XXXX.conf

+ test -z ''

+ test -d /usr/local/etc/vservers/XXXX

+ test -e /usr/local/etc/vservers/XXXX/name

+ read VSERVER_NAME

+ test start '!=' start -o -n ''

+ isAvoidNamespace /usr/local/etc/vservers/XXXX

+ local cfgdir

+ /usr/local/sbin/vserver-info - FEATURE namespace

++ /usr/local/sbin/vserver-info /usr/local/etc/vservers/XXXX CFGDIR

+ cfgdir=/usr/local/etc/vservers/XXXX

+ test '!' -e /usr/local/etc/vservers/XXXX/namespace

+ test -e /usr/local/etc/vservers/.defaults/nonamespace -o -e /usr/local/etc/vservers/XXXX/nonamespace

+ exec /usr/local/sbin/vnamespace new /usr/local/sbin/vserver --nonamespace debug XXXX start

wait(): No child processes

SUCCESS!

[ not all stages completely necessary but this is all of the fixes suggested so far and it's what I ran... ]

1. kernel 2.6.11.9 straight from kernel.org patch with 2.6.11.9-rc1 and config from a Debian 2.6.8 kernel compile, run

2. dietlibc-0.28 vanilla from their main site with the following changes

http://vserver.13thfloor.at/Stuff/DIETLIBC/dietlibc-gethostbyname-fix01.diff

http://vserver.13thfloor.at/Stuff/DIETLIBC/dietlibc-syscall-fix01.diff

Comment out #ifdef WANT_THREAD_SAFE from dietfeatures.h

Add .register %g2,#scratch to libcompat/syscall.S

Change ".align 4" to ".align 8" for sys_errlist in sparc64/errlist.S

compile in 32 bit mode (i.e. sparc32 make )

3. vanilla util-vserver-0.30.207 from distribution site

replace lib/syscall-alternative.h with http://vserver.13thfloor.at/Experimental/SYSCALL/syscall.h

export DIET=~/where/ever/you/built/it

sed -i "s/enable_val/enableval/" configure

./configure

make

make install

4. echo "/usr/local/lib/util-vserver/vshelper" >/proc/sys/kernel/vshelper

5. /usr/local/sbin/vserver

nonamespace XXXX start