Preventing ip conflicts in xen

By branko posted on September 22nd, 2008

So lately I’m playing with xen kernel and virtualization, and I came across one relatively big problem. Let’s say i want to share my guest machines to… let’s say clients. You must give them root… because that’s whats vps-es all all about… having root access to OS without having to purchase expensive hardware. So having that in mind they are by default untrusted and unpredictable, they can do god knows what in there!

So what caught my eye?

By default xen, and all those tools, cpanels and stuff don’t really have a way of sorting out ip conflicts. So basically you have some bunch of scripts that will setup clients ip address during his machine startup, you have vif ip statment in vm_xen.conf file. But what’s really holds clients from entering:

ifconfig eth0 xxx.xxx.xxx.xxx 

(where xxx is ip of some super important server in same netmask). Luckily for me I came across this problem while still in testing, but here’s what I found and came up with after 3 days intensive googling.

So basically xen supports ip declaration in vif statment of dom config file like this:

vif = ['ip=xxx.xxx.xxx.xxx, more parametars here....']

Also you can declare multiple ip's by simply putting space between them, like this:

vif = ['ip=xxx.xxx.xxx.xx1 xxx.xxx.xxx.xx2, more parametars here....']

For the purpose of ip conflict prevention make sure you declare unique mac address in vif section to.

So what does this ip thingy in vif do? Absolutely nothing! (at least not yet )

Next step is to install ebtables (http://ebtables.sourceforge.net/) on your distro. After that all we need to do i patch up a vif-bridge script located in /etc/xen/scripts/

So here’s the diff:

[root@vps scripts]# diff -u vif-bridge-org vif-bridge
--- vif-bridge-org      2008-07-30 21:26:16.000000000 +0200
+++ vif-bridge  2008-07-30 21:30:59.000000000 +0200
@@ -57,15 +57,35 @@
online)
setup_bridge_port "$vif"
add_to_bridge "$bridge" "$vif"
-        ;;
-
+        ebtables -N $vif
+              ebtables -P $vif DROP
+              ebtables -A INPUT -i $vif -j $vif
+              ebtables -A FORWARD -i $vif -j $vif
+              ebtables -A $vif -p ARP --arp-opcode 1 -j ACCEPT
+
+             if [ ! -z "$ip" ]
+             then
+               for oneip in $ip
+               do
+                 ebtables -A $vif -p IPv4 --ip-src $oneip -j ACCEPT
+                 ebtables -A $vif -p IPv4 --ip-dst $oneip -j ACCEPT
+                 ebtables -A $vif -p ARP --arp-opcode 2 --arp-ip-src $oneip -j ACCEPT
+               done

+                 ebtables -A $vif --log-prefix="arp-drop" --log-arp -j DROP

+              fi
+       ;;
+
offline)
do_without_error brctl delif "$bridge" "$vif"
do_without_error ifconfig "$vif" down
-        ;;
+        do_without_error ebtables -D INPUT -i $vif -j $vif
+        do_without_error ebtables -D FORWARD -i $vif -j $vif
+        do_without_error ebtables -F $vif
+        do_without_error ebtables -X $vif
+       ;;
esac-handle_iptable
+#handle_iptable
log debug "Successful vif-bridge $command for $vif, bridge $bridge."

if [ "$command" == "online" ]

Presuming you use bridging scripts this effectively binds ip address(es) from "vif = ['ip=xxx.xxx.xxx.xxx']" list to mac addresses from vif list. Binding is done while enabling vps machine and undone when powering it off. So this way untrusted user is limited only to the ip addresses defined in xen guest conf file, trying to change existing ip address into another ip on network will only render that machine unresponsive.