CloudClient

Introduction

Many folks ask how they can use Security Onion to monitor and defend their cloud environments. Most cloud environments don't provide anything like a tap or span port, but we can use daemonlogger or netsniff-ng as a virtual tap. This virtual tap will copy all traffic from our production cloud box to an OpenVPN bridge that transports the traffic to our Security Onion sensor where it is then analyzed.

Traffic Flow and NIC offloading functions

The cloud client uses daemonlogger or netsniff-ng to copy all packets from eth0 to tap0 (OpenVPN). OpenVPN transports the packets to the cloud sensor, where tap0 is a member of bridge br0. The standard Security Onion stack sniffs br0. NIC offloading functions must be disabled on all of these interfaces (eth0 and tap0 on cloud client, and tap0 and br0 on cloud sensor) to ensure that Snort, Bro, etc. all see traffic as it appeared on the wire. This guide will walk you through disabling NIC offloading functions on eth0 and br0 via /etc/network/interfaces and tap0 via /etc/openvpn/up.sh.

Daemonlogger vs netsniff-ng

This guide is written using daemonlogger because it is more likely to be available on most cloud boxes. If netsniff-ng is available, it can provide higher performance (less packet loss), and you would just need to change the calls from daemonlogger to netsniff-ng and translate the options to the equivalent netsniff-ng options.

References and Thanks

This is based on Josh Brower's great work:

http://www.slideshare.net/DefensiveDepth/so-conference-2014

http://www.sans.org/reading-room/whitepapers/detection/security-onion-cloud-client-network-security-monitoring-cloud-34335

The OpenVPN configuration shown below is based on the following guides:

https://help.ubuntu.com/community/OpenVPN

https://help.ubuntu.com/lts/serverguide/openvpn.html

Warning!

This cloud client is considered experimental. USE AT YOUR OWN RISK!

This guide was written for Security Onion 12.04 and is currently being updated to work with Security Onion 14.04. It may not be fully functional until it is fully updated.

Security Onion Sensor

We first start with our Security Onion sensor. Run Security Onion Setup Phase 1 (Network Configuration), allow it to write your /etc/network/interfaces file, but DON'T reboot at the end:

sudo sosetup

Add br0 to /etc/network/interfaces and disable offloading functions:

cat << EOF | sudo tee -a /etc/network/interfaces
# Bridge for OpenVPN tap0
auto br0
iface br0 inet manual
  bridge_ports none
  post-up for i in rx tx sg tso ufo gso gro lro; do ethtool -K \$IFACE \$i off; done
EOF

Reboot:

sudo reboot

Run Security Onion Setup Phase 2 and choose to monitor br0:

sudo sosetup

Setup has locked down the UFW firewall, so let's go ahead and allow OpenVPN port 1194:

sudo ufw allow 1194

Install OpenVPN:

sudo apt-get update
sudo apt-get install openvpn easy-rsa

Next, copy files to the /etc/openvpn/easy-rsa/ directory:

sudo mkdir /etc/openvpn/easy-rsa/ 
sudo cp -r /usr/share/easy-rsa/* /etc/openvpn/easy-rsa/

Edit /etc/openvpn/easy-rsa/vars:

sudo vi /etc/openvpn/easy-rsa/vars

Change these lines at the bottom so that they reflect your new CA:

export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"
Also change any lines that contain "changeme"

Setup the CA and create the first server certificate:

cd /etc/openvpn/easy-rsa/ ## move to the easy-rsa directory
sudo chown -R root:sudo .  ## make this directory writable by the system administrators
sudo chmod g+w . ## make this directory writable by the system administrators
source ./vars ## execute your new vars file
./clean-all  ## Setup the easy-rsa directory (Deletes all keys)
./build-dh  ## takes a while consider backgrounding
./pkitool --initca ## creates ca cert and key
./pkitool --server server ## creates a server cert and key
## If you get this error: 
##    "The correct version should have a comment that says: easy-rsa version 2.x"
## Try This:
##     sudo ln -s openssl-1.0.0.cnf openssl.cnf
## Refer to: https://bugs.launchpad.net/ubuntu/+source/openvpn/+bug/998918
cd keys
openvpn --genkey --secret ta.key  ## Build a TLS key
sudo cp server.crt server.key ca.crt dh2048.pem ta.key ../../
# The Certificate Authority is now setup and the needed keys are in /etc/openvpn/

Create a script that OpenVPN will call when the tunnel comes up to add tap0 to br0 and disable offloading functions on tap0:

cat << EOF | sudo tee -a /etc/openvpn/up.sh
#!/bin/sh

BR=\$1
DEV=\$2
/sbin/ip link set "\$DEV" up promisc on
/sbin/brctl addif \$BR \$DEV

for i in rx tx sg tso ufo gso gro lro; do ethtool -K \$DEV \$i off; done
EOF

Create a script that OpenVPN will call when the tunnel goes down:

cat << EOF | sudo tee -a /etc/openvpn/down.sh
#!/bin/sh

BR=\$1
DEV=\$2

/sbin/brctl delif \$BR \$DEV
/sbin/ip link set "\$DEV" down
EOF

Make both of these scripts executable:

sudo chmod +x /etc/openvpn/up.sh /etc/openvpn/down.sh

Create OpenVPN server.conf:

sudo cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/
sudo gzip -d /etc/openvpn/server.conf.gz

Modify /etc/openvpn/server.conf:

sudo sed -i 's|^dev tun$|;dev tun|g' /etc/openvpn/server.conf
sudo sed -i 's|^;dev tap|dev tap|g' /etc/openvpn/server.conf
sudo sed -i 's|^comp-lzo|;comp-lzo|g' /etc/openvpn/server.conf

cat << EOF | sudo tee -a /etc/openvpn/server.conf

up "/etc/openvpn/up.sh br0"
down "/etc/openvpn/down.sh br0"
EOF

Restart OpenVPN server:

sudo service openvpn restart

Check log for errors:

tail -f /var/log/syslog

Verify tap0 came up:

ifconfig

Generate client cert (replacing "client" with the name of the cloud client you want to add):

cd /etc/openvpn/easy-rsa/ ## move to the easy-rsa directory
source ./vars             ## execute the vars file
KEY_CN=client ./pkitool client

Copy generated files to cloud client:

scp /etc/openvpn/easy-rsa/keys/client* username@hostname:~/
scp /etc/openvpn/easy-rsa/keys/ca.crt username@hostname:~/

Cloud client

Install openvpn and daemonlogger:

sudo apt-get update
sudo apt-get install openvpn daemonlogger

Copy crt files to /etc/openvpn/:

sudo cp client* /etc/openvpn/
sudo cp ca.crt /etc/openvpn/

Create OpenVPN client.conf:

sudo cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/

Modify /etc/openvpn/client.conf:

sudo sed -i 's|^dev tun$|;dev tun|g' /etc/openvpn/client.conf
sudo sed -i 's|^;dev tap|dev tap|g' /etc/openvpn/client.conf
sudo sed -i 's|^comp-lzo|;comp-lzo|g' /etc/openvpn/client.conf

cat << EOF | sudo tee -a /etc/openvpn/client.conf

up "/etc/openvpn/up.sh"
down "/etc/openvpn/down.sh"
EOF

Find the "remote my-server-1 1194" line in /etc/openvpn/client.conf and replace my-server-1 with the hostname or IP address of your OpenVPN server.

Create a script that OpenVPN will call when the tunnel comes up to disable offloading functions on tap0 and start daemonlogger. The daemonlogger BPF at minimum should exclude the OpenVPN traffic on port 1194 ('not port 1194'). You may need to restrict this BPF even further if there is other traffic you do not wish to send across the OpenVPN tunnel.

cat << EOF | sudo tee -a /etc/openvpn/up.sh
#!/bin/sh

IN=eth0
OUT=\$1

daemonlogger -d -i \$IN -o \$OUT 'not port 1194'

for i in rx tx sg tso ufo gso gro lro; do ethtool -K \$OUT \$i off; done
EOF

Create a script that OpenVPN will call when the tunnel goes down:

cat << EOF | sudo tee -a /etc/openvpn/down.sh
#!/bin/sh

pkill daemonlogger
EOF

Make both of these scripts executable:

sudo chmod +x /etc/openvpn/up.sh /etc/openvpn/down.sh

Restart OpenVPN client:

sudo service openvpn restart

Check log for errors:

tail -f /var/log/syslog

Verify that tap0 came up:

ifconfig

Disable NIC offloading functions on main ethernet interface. Add the following to your eth stanza in /etc/network/interfaces:

  post-up for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done

Check traffic

Your Security Onion sensor should now be seeing traffic from your Cloud Client. Verify as follows:

sudo tcpdump -nnvvAi tap0

tap0 should be a member of br0, so you should see the same traffic on br0:

sudo tcpdump -nnvvAi br0

When you ran Setup phase 2 you configured Security Onion to monitor br0, so you should be getting IDS alerts and Bro logs.

Hardening

Once you get everything working properly, you should configure OpenVPN (server and client) and daemonlogger to run as a limited user.

Tuning

If your cloud box is seeing lots of traffic, daemonlogger may not be able to keep up, resulting in packet loss. You may need to switch to netsniff-ng for higher performance. Don't forget to run netsniff-ng as a limited user!