the virtuozzo admin

If you attempt to start a container after a hardware node crash, you may see an error like this:

Starting Container ...
vzquota : (warning) Incorrect quota shutdown for id 101, recalculating disk usage
vzquota : (error) quota check : lstat `somefile.ext': No such file or directory
Running vzquota on failed for Container 101 [1]

This means that the container’s quota system is crashed. You can restart it simply switching the quota off and on:

vzquota off 101
vzquota on 101

The container should now start up properly.

When troubleshooting load issues on a node, top is still an effective command for a quick glance at what’s going on. However, since the PIDs for processes in a container do not correlate with users on the parent node, it can sometimes be unclear as to what container is running the processes.

Virtuozzo has a built-in command called vzpid which identifies the container running a given process. For example, if I wanted to find out which container was running process ID #31299:

[root@server ~]# vzpid 31299
Pid    VEID    Name
31299    102    mysqld

When starting up a VPS, you may see an error like this:

vzquota : (error) Quota on syscall for 101: Device or resource busy

In more detail:

Starting Container ...
vzquota : (error) Quota on syscall for id 1: Device or resource busy
vzquota : (error)     Possible reasons:
vzquota : (error)     - Container's root is already mounted
vzquota : (error)     - there are opened files inside Container's private area
vzquota : (error)     - your current working directory is inside Container's
vzquota : (error)       private area
vzquota : (error)     Use -v option to see currently opened file(s).
Running vzquota on failed for Container 1 [3]
Container start failed

If this happens, run a grep to see if any processes are running from inside the container’s private folder (replace ’1′ with your VPS id)

lsof 2> /dev/null | egrep '/vz/root/1/|/vz/private/1/'

Kill of any processes this returns, and restart the container.

I came across a VPS that was recently moved to another node and noticed that Apache wasn’t starting due to missing modules.  When I saw that the modules folder in Apache was linked to /usr/lib64, I checked that folder to see that the httpd directory was there, but not really “there”. It looked like this:

?---------  ? ?    ?       ?            ? httpd

Question marks in the file name and descriptor, clearly indicating that the VPS container was missing something. Well, I went onto the parent node and into /vz/private/VEID/fs/root/usr/lib64 and the files were there, along with some strange broken symlink that looks like this:

lrwxrwxrwx   1 root root   72 Mar  8  2009 ._vzlnk_. -> ////centos/5/x86_64/httpd-2.2.3-11.el5_2.centos.4.x86_64/usr/lib64/httpd

To fix one instance, I removed the broken symlink and that fixed the problem. I also tried to fix this on another container by copying over the httpd folder in the lib64 directory and that did the trick as well, but I assume it’s the broken symlink that caused the problem.

Further troubleshooting indicated that the container was somehow using a different OS template than what it was on the previous server, which was causing Virtuozzo to be unable to find some files. If this is the case, you can edit /etc/vz/conf/$VEID.conf and change the ostemplate value to what it should be, then restart the container.

If that doesn’t fix it, you’ll probably still need to remove all those bad symlinks. Here’s a simple script stolen from here that should remove all these for you:

#!/bin/bash
VEID=101 # Replace with your container ID
for i in `find /vz/private/$VEID/fs/root/ -noleaf -type l -print | perl -nle ‘-e || print’|grep ._vzlnk_.`
do
ln -sf `ls -ga $i|grep ._vzlnk_.|awk ‘{print $10}’|sed ‘s/\/\/\//\/vz\/template/’` $i
done

When executing commands on a Redhat-family VPS (Such as Fedora or CentOS) you may get the following error when executing commands via sudo:

audit_log_user_command(): Connection refused

All this means is that a feature for audit logging is disabled in the Linux kernel, and it’s nothing to be concerned about. However, it seems that no one knows how to get rid of it…

If your service container isn’t working or you need to recreate it, follow these steps:

1) Download the autoinstaller for Virtuozzo:

wget http://download.parallels.com/virtuozzo/virtuozzo4.0/linux/vzinstall-linux.bin

2) Run the autoinstaller and select the download only option. This will download the files to /root/virtuozzo

chmod 755 vzinstall-linux.bin
./vzinstall-linux.bin

3) Extract the source to a temporary location:

mkdir -p /vz/temp
/root/virtuozzo/download/Linux/{arch}/virtuozzo-{version}-{arch}.sfx -d /vz/temp--extract

4) Now create the service container and assign it an IP:

vzsveinstall -v -D /vz/temp -s $ip

If you’re installing APF on a VPS, chances are you’ll need to make a couple changes to your APF’s config file. Here are a couple errors that are common:

eth0: error fetching interface information: Device not found

You need to change the interface values in apf.conf to use venet0 instead of eth0 since the VPS is on a virtual interface:

IFACE_IN="eth0"
IFACE_OUT="eth0"

unable to load iptables module (ip_tables), aborting.

Change the MONOKERN value from ’0′ to ’1′ .

Sometimes you may need to remove Virtuozzo, either to reinstall or just remove it, but are not able to completely reinstall the OS. I’ll stress though that if possible, you should reinstall the OS entirely to completely remove VZ. But in case you can’t, here are the steps to uninstall:

1. Stop Virtuozzo

service vz stop

2. Uninstall all Virtuozzo packages

rpm -qa|grep swsoft|grep vz|xargs yum remove -y
rpm -qa|grep swsoft|grep -E “virtu|-ez-|-tmpl-”|xargs yum remove -y

3. Remove contents from /vz : actionlog, backups, lock, private, root, template, vzup2date

cd /vz/ && mkdir _before_reinstall && mv * _before_reinstall

I usually have /vz on a separate partion, so I move all its contents to another partition on the server and just reformat using mkfs.

4. Remove configs from /etc

mkdir -p /vz/_before_reinstall/etc && find /etc -name vz\* -exec mv {} /vz/_before_reinstall/etc \;

5. Check /etc/grub.conf and make sure the boot kernel is standard (meaning, the parent OS and not VZ)

6. Reboot the server