toic.org - Entries for the tag xen

Firewalling xen bridge

Sun, 19 Apr 2009 19:52:29 +0000

Occasionally you will wish to block certain ports to your DomUs from Dom0. By default you wish to allow any traffic from and to DomU but for some security considerations, I found it to be wise to block some ports to and from my clients DomUs. One such port range is for example IRC. Although it can be routed trough alternate ports, most of automated malicious scripts use default ones. It's quite handy to block them so they ain't able to contact home.

As said by default Xen bridge is open for all traffic from and to DomUs. It's up to DomU admin to firewall their own virtual machine. Unfortunately some just forget to do the proper securing of the system, and as a result you get compromised DomU contacting various botnets, and executing all kind of nasty stuff.

To prevent this we can make a firewall rules in DomU that will by default block some traffic. Since I'm using bridged network, firwalling must be done on bridge. I found this great article on shorewall manuals explaining how to setup bridged network firewall. I installed it and tested it on 32bit Centos 5.2, but it should work on any system.

Fist of all you will need to download and install latest shorewall.

As stated in documentation link above, you must enable bridge support in shorewall.conf:

nano /etc/shorewall/shorewall.conf

Set:

BRIDGING=Yes

Now we have to edit our firewall zones:

nano /etc/shorewall/zones

It should look something like this:

#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw    firewall
dom0 ipv4
domU ipv4
net ipv4
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE

Next thing to do is to define network interfaces, we will be dealing with two network interfaces: virtualized eth0 and bridge:

nano /etc/shorewall/interfaces

And the file should look like this:

#ZONE INTERFACE    BROADCAST    OPTIONS
-    xenbr0 - dhcp
net eth0 detect dhcp
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

Next stop, hosts file:

nano /etc/shorewall/hosts

And the file should look like this:

#ZONE HOST(S) OPTIONS
dom0 xenbr0:vif0.0
domU xenbr0:vif+     routeback
net xenbr0:peth0
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS LINE -- DO NOT REMOVE

Now let's make some policies in our firewall:

nano /etc/shorewall/policy

And the file should look like this:

#SOURCE DEST POLICY LOG LIMIT: CONNLIMIT:
# LEVEL BURST MASK
fw    all ACCEPT
all fw    ACCEPT info
dom0 all ACCEPT
all dom0 ACCEPT info
domU all ACCEPT
all domU     ACCEPT
net net NONE
all all REJECT info
#LAST LINE -- DO NOT REMOVE

This will by default allow any traffic through the bridge. You can also specify DROP policy for your Dom0 and then open necessary ports in rules file. Note that the fw and dom0 are the same but they both need to be declared in policy and rules file. So... for now, this does not block IRC traffic as we started to do, all we need to do now is to setup the rules file:

nano /etc/shorewall/rules

And the file should look like this:

#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT    TIME PORT PORT(S) DEST LIMIT GROUP
#irc
REJECT net domU tcp 6660:6669
REJECT domU net tcp 6660:6669
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

After adding this, it will block all incoming and outgoing traffic from port range 6660 to 6669 for all DomUs. If you wish to add an exception to one DomU you can simply edit the rules file and insert the exception above the REJECT:

#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT    TIME PORT PORT(S) DEST LIMIT GROUP
#DomU exceptions
ACCEPT net domU:192.168.0.10    tcp 6660:6669
ACCEPT domU:192.168.0.10    net tcp 6660:6669
#DomU restrictions
#irc
REJECT net domU tcp 6660:6669
REJECT domU net tcp 6660:6669
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

This way only the DomU with ip 192.168.0.10 will have unblocked IRC ports. Although the above config should work it didn't for me. Centos 5.2 by default comes with:

net.bridge.bridge-nf-call-iptables = 0

so no bridge firewalling is actually done. To enable this edit your sysctl.conf file:

nano /etc/sysctl.conf

and append:

net.bridge.bridge-nf-call-iptables = 1

now run:

sysctl -p

And the bridged firewall for your DomUs should work now.

Multiple network interfaces in Xen

Mon, 06 Oct 2008 19:44:41 +0000

By default xen tools comes with only one network interface enabled for your dom0 and domU machines. So what if you want to add some more? It's actually very simple.

All you need to do is run:

/etc/xen/scripts/network-bridge start vifnum=1 netdev=eth1 bridge=xenbr1

This will effectively create one extra xenbr attached to eth1 interface. You can repeat the above command for all your interfaces, and you can stop them in the same manner, just replace start with stop.

To enable this automatically you can create file named, multi-network-bridge:

vim /etc/xen/scripts/multi-network-bridge

paste this:

#!/bin/sh
/etc/xen/scripts/network-bridge $@ vifnum=0 netdev=eth0 bridge=xenbr0
/etc/xen/scripts/network-bridge $@ vifnum=1 netdev=eth1 bridge=xenbr1

Of course you can add up as many interfaces you like in this script. After you have added your interfaces, you need to edit xend-config.spx file:

vim /etc/xen/xend-config.spx

Find a line defining network script, it should by default look like this:

(network-script network-bridge)

Edit it so it contains your newly created multi network bridge script. In my case it should look like:

(network-script multi-network-bridge)

All you need to do now is restart xend service:

/etc/init.d/xend restart

New network bridge named xenbr1 should be available now.

How can I add up another network bridge to my domU machine?

It's pretty simple actually, you already have defined vif statements for current network interface, all you need to do now is edit that domU config file, precisely vif line and add up another xenbr interface.

For example, if my domU vif line looks like this:

vif = ['ip=xx.xxx.167.4, vifname=vifbran0, rate = 10000KB/s, bridge=xenbr0']

with another xenbr interface it should look like this:

vif = ['ip=89.201.167.4, vifname=vifbran0, rate = 10000KB/s, bridge=xenbr0', 'ip=192.168.1.10, vifname=vifbran1, rate = 10000KB/s, bridge=xenbr1']

Also if you wish to use ip conflict prevention you must add additional mac section in vif configuration. Restart your domU and voila, another network interface is present

Preventing ip conflicts in xen

Mon, 22 Sep 2008 19:07:40 +0000

Lately I was playing with stock xen kernel and virtualization, and I came across one relatively big problem. Let’s say I want to share my guest machines to, let’s say clients. You must give them root… because that’s whats VPS-es all all about, having root access to OS without having to purchase expensive hardware. Having that in mind they are by default untrusted and unpredictable, they can do god knows what in there!

So what caught my eye?

By default xen, and available management tools, don’t really have a way of sorting out IP conflicts in bridged mode. Basically you have bunch of scripts that will provision VPS alongside with IP address. Looking at the conf files you have vif and IP declarations in vm_xen.conf file.

But what is really preventing clients from entering:

ifconfig eth0 xxx.xxx.xxx.xxx

where xxx is ip of some super important server in same netmask?

Luckily, I came across this problem while still in testing, here’s what I found and came up with after 3 days intensive googling.

Xen supports IP declaration in vif statment of domU config file like this:

vif = ['ip=xxx.xxx.xxx.xxx, more parametars here....']

Also you can declare multiple IP's by simply putting space between them, like this:

vif = ['ip=xxx.xxx.xxx.xx1 xxx.xxx.xxx.xx2, more parametars here....']

For the purpose of IP conflict prevention make sure you declare unique mac address in vif section to.

So what does this IP thingy in vif do? Absolutely nothing (at least not yet)!

Next step is to install ebtables (http://ebtables.sourceforge.net/) on your distro. Then all we need to do is patch up a vif-bridge script located in /etc/xen/scripts/

So here’s the diff:

--- vif-bridge-org 2008-07-30 21:26:16.000000000 +0200
+++ vif-bridge 2008-07-30 21:30:59.000000000 +0200
@@ -57,15 +57,35 @@
online)
setup_bridge_port "$vif"
add_to_bridge "$bridge" "$vif"
- ;;
-
+ ebtables -N $vif
+ ebtables -P $vif DROP
+ ebtables -A INPUT -i $vif -j $vif
+ ebtables -A FORWARD -i $vif -j $vif
+ ebtables -A $vif -p ARP --arp-opcode 1 -j ACCEPT
+
+ if [ ! -z "$ip" ]
+ then
+ for oneip in $ip
+ do
+ ebtables -A $vif -p IPv4 --ip-src $oneip -j ACCEPT
+ ebtables -A $vif -p IPv4 --ip-dst $oneip -j ACCEPT
+ ebtables -A $vif -p ARP --arp-opcode 2 --arp-ip-src $oneip -j ACCEPT
+ done
+ ebtables -A $vif --log-prefix="arp-drop" --log-arp -j DROP
+ fi
+ ;;
+
offline)
do_without_error brctl delif "$bridge" "$vif"
do_without_error ifconfig "$vif" down
- ;;
+ do_without_error ebtables -D INPUT -i $vif -j $vif
+ do_without_error ebtables -D FORWARD -i $vif -j $vif
+ do_without_error ebtables -F $vif
+ do_without_error ebtables -X $vif
+ ;;
esac-handle_iptable
+#handle_iptable
log debug "Successful vif-bridge $command for $vif, bridge $bridge."
if [ "$command" == "online" ]

Asuming you use bridging scripts this effectively restricts IP address(es) from "vif = ['ip=xxx.xxx.xxx.xxx']" list to mac addresses in vif list. Restrictoins are done while booting up VPS and removed when powering it off. This way untrusted user is limited only to IP addresses defined in xen guest conf file. Trying to change existing IP address into another IP on network will only render that machine unresponsive.