[KLUG Members] Looking for advice on which virtualization technology to use for production systems

[klug at obbink.eu]

Hi Adam,

We are using openvz for some time now (about 150 servers now). It works very nice.
We use centos 5 as basic os on what we call the hardware node. On top of that we place the image of a server.
You can build different images that can be placed on the hardware node. If your hardware is powerful enough you can run
several images on the same node.

Advantage of openvz is that if the firewall is set up correctly, in theory, there is a smaller change of being hacked.
The /proc for instance is not available on the vz-server.
It is also possible to do an on the fly migrate of a vz-server to another hwnode. Very busy vz's often fail however when migrated running.

(Dis?)-advantage is you can only run linux images. Those images don't necessary have to be centos. Debian or alike are possible.

Consider that even on very powerful hardware you can't run a large number of images. The 'bottle-neck' is not your processor or the amount of memory available.
The bottle-neck will be your harddisks. Most harddisks can do something of 300 head moves a second, with 10 images each running 30 processes you have 1 head move per second per process left.
Your server load will be nice, your free memory will be nice and you may be wondering why everything isn't preforming like you suspected.
On xeon servers we run up to 10 images. Depending on the suspected load.

I can't supply you with an image because of the (licensed) software we have installed on it. Same reason why I can't supply a final installation script.
I have added some scripts you can use to create a server. You have to do some editing though.
This script disables se-linux. It is possible however to run with se-linux enabled. See the open-vz site for more info.
Don't use the scrips for life systems without proper checking!


Happy hacking.

Bert.



-------------- next part --------------
#
# example script, please do not use without editing !!!
#
#--------------------------------------------- script 1 ---------------------------------

echo open VZ install
###############
cd /etc/yum.repos.d
rm openvz.repo
wget http://download.openvz.org/openvz.repo
rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ

echo kernel upgrade
###############
cd /root
rm kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
wget http://download.openvz.org/kernel/branches/2.6.18/028stab035.1/kernel-smp-2.6.18-ovz028stab035.1.i686.rpm
rpm -ivh kernel-smp-2.6.18-ovz028stab035.1.i686.rpm

echo Grub aanpassen
###############
mv /boot/grub/grub.conf /boot/grub/grub.conf.original
cat /boot/grub/grub.conf.original | sed 's/default=1/default=0/g' | sed 's/Fedora\ Core/OpenVZ/g' >  /boot/grub/grub.conf

echo sysctl aanpassen
###############
mv /etc/sysctl.conf /etc/sysctl.conf.original
touch /etc/sysctl.conf
echo "# On Hardware Node we generally need" >> /etc/sysctl.conf
echo "# packet forwarding enabled and proxy arp disabled" >>  /etc/sysctl.conf
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
echo "net.ipv4.conf.default.proxy_arp = 0" >> /etc/sysctl.conf
echo "# Enables source route verification" >> /etc/sysctl.conf
echo "net.ipv4.conf.all.rp_filter = 1" >> /etc/sysctl.conf
echo "# Enables the magic-sysrq key" >> /etc/sysctl.conf
echo "kernel.sysrq = 1" >> /etc/sysctl.conf
echo "# TCP Explict Congestion Notification" >> /etc/sysctl.conf
echo "#net.ipv4.tcp_ecn = 0" >> /etc/sysctl.conf
echo "# we do not want all our interfaces to send redirects" >> /etc/sysctl.conf
echo "net.ipv4.conf.default.send_redirects = 1" >> /etc/sysctl.conf
echo "net.ipv4.conf.all.send_redirects = 0" >> /etc/sysctl.conf

echo selinux aanpassen
###############
mv /etc/sysconfig/selinux  /etc/sysconfig/selinux.original 
cat /etc/sysconfig/selinux.original | sed 's/SELINUX\=enforcing/SELINUX\=disabled/g' > /etc/sysconfig/selinux

echo screen installeren
###############
cd /root/tools
yum -y install screen

echo reboot
###############
reboot

#--------------------------------------------- script 2 ---------------------------------
# part 2

echo -n "Type server te installeren [dds|shared] : "
read getimage

#setting server time
echo configuring time
echo "10 3 * * * root /usr/bin/rdate -s time.xs4all.nl" >> /etc/crontab
/usr/bin/rdate -s time.xs4all.nl
echo "/usr/bin/rdate -s time.xs4all.nl" >> /etc/rc3.d/S99local

echo installatie VZ tools
###############
yum -y install vzctl vzquota
service vz start
yum -y install vzpkg vzyum vzrpm43-python vzrpm44-python
yum -y install vztmpl-fedora-core-5
vzpkgcache fedora-core-5


#aanpassen iptables
mv /etc/sysconfig/iptables /etc/sysconfig/iptables.old
cp /root/tools/iptables_nieuw /etc/sysconfig/iptables

echo Restart VZ server, broken pipe error geen probleem
###############
service vz restart

###############
echo Perl installatie
yum -y install perl 
echo VZdump install
cd /root
rm vzdump-0.4-1.noarch.rpm
wget http://www.proxmox.com/cms_proxmox/cms/upload/vzdump/vzdump-0.4-1.noarch.rpm
rpm -Uvh vzdump-0.4-1.noarch.rpm

###############
if [ $getimage == "dds" ]; then
	wget -O /root/vzdump-image.tar $INSTALL/dds-image.tar
elif [ $getimage == "shared" ]; then
        wget -O /root/vzdump-image.tar $INSTALL/shared-image.tar
else 
	wget -O /root/vzdump-image.tar $INSTALL/default-image.tar
fi

## herstarten FW rules
iptables -F;/etc/init.d/iptables restart

## Reboot ? 
echo -n "Server herstarten?";
read rebootanswer

if [ $rebootanswer = "yes" ]; then
	echo rebooting server
	reboot
else
	echo "You may always reboot yourself"
fi

#-------------------------------------- script 3 -----------------------------
# Part 3
echo -n "Geef dds veid:"
read veid
echo restoring DDS image 
vzdump --restore /root/vzdump-image.tar $veid
#	vzctl set $veid --hostname $vzservernaam --save
	vzctl set $veid --nameserver $NAMESERVER --save
	vzctl set $veid --quotaugidlimit 2000 --save
	vzctl set $veid --diskspace 25G:25G --save
	vzctl set $veid --kmemsize 2147483647:2147483647 --save
	vzctl set $veid --lockedpages 2147483647:2147483647 --save
	vzctl set $veid --privvmpages 2147483647:2147483647 --save
	vzctl set $veid --shmpages 2147483647:2147483647 --save
	vzctl set $veid --numproc 2147483647:2147483647 --save
	vzctl set $veid --physpages 2147483647:2147483647 --save
	vzctl set $veid --vmguarpages 2147483647:2147483647 --save
	vzctl set $veid --oomguarpages 2147483647:2147483647 --save
	vzctl set $veid --numtcpsock 2147483647:2147483647 --save
	vzctl set $veid --numflock 2147483647:2147483647 --save
	vzctl set $veid --numpty 2147483647:2147483647 --save
	vzctl set $veid --numsiginfo 2147483647:2147483647 --save
	vzctl set $veid --tcpsndbuf 2147483647:2147483647 --save
	vzctl set $veid --tcprcvbuf 2147483647:2147483647 --save
	vzctl set $veid --othersockbuf 2147483647:2147483647 --save
	vzctl set $veid --dgramrcvbuf 2147483647:2147483647 --save
	vzctl set $veid --numothersock 2147483647:2147483647 --save
	vzctl set $veid --dcachesize 2147483647:2147483647 --save
	vzctl set $veid --numfile 2147483647:2147483647 --save
	vzctl set $veid --numiptent 2147483647:2147483647 --save
	vzctl set $veid --diskinodes 2147483647:2147483647 --save
	vzctl set $veid --numiptent 400 --save
	
	echo "ALLOWREBOOT=\"yes\"" >> /etc/sysconfig/vz-scripts/$veid.conf

#------------
	echo
	echo Meer info over migreren op open vz check\: http\:\/\/wiki.openvz.org\/Migration_from_one_HN_to_another
	echo