Table 1 / Figure 1

low-latency/figures/motivation/pcap/all-filtered.pdf.svg

Steps to reproduce measurements

Images

Images used for testing

Device under test, evaluator

Linux machine 4.19.12 #1 SMP Fri Feb 1 21:28:58 CET 2019 x86_64 GNU/Linux

Loadgen, timestamper

Linux machine 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64 GNU/Linux

Setup

Setup loadgen

git clone https://github.com/gallenmu/MoonGen /root/moongen
cd /root/moongen
git checkout 0450d0f84ac900843031b61a9f28a8194add147d 
/root/moongen/build.sh
/root/moongen/bind-interfaces.sh
/root/moongen/setup-hugetlbfs.sh

Setup device under test (DuT):

Boot VM host parameter

vga=791 boot=live dhcp BOOTIF=01-1a-2b-3c-4e-5f-67 random.trust_cpu=on intel_iommu=on noprompt lvm hostname=vmhost noeject

Setup VM host

echo 'deb http://deb.debian.org/debian experimental main contrib non-free
deb-src http://deb.debian.org/debian experimental main contrib non-free' >> /etc/apt/sources.list

apt update
apt install -t experimental -y virt-manager qemu-system pkg-config libvirt-dev python-libvirt

ip link set up dev eno7
ip link set up dev eno8

# disable ipv6 to avoid pf chatting on vf connection
sysctl net.ipv6.conf.eno7.disable_ipv6=1
sysctl net.ipv6.conf.eno8.disable_ipv6=1

# download network configs
git clone https://github.com/gallenmu/low-latency /root/low-latency

# configure mgmt interface of VM
virsh net-define /root/low-latency/configs/vm/net.xml
virsh net-start net

# configure measurement interfaces of VM
virsh net-define /root/low-latency/configs/vm/net7.xml
virsh net-define /root/low-latency/configs/vm/net8.xml
virsh net-start net7
virsh net-start net8

virt-install --cpu host-passthrough --memory 16384 --vcpus=3 --cpuset=1-3 --boot=network \
--name vm --nodisks --network="network=net,mac=11:22:33:44:55:66,model=virtio" \
--network="network=net7,mac=52:54:00:8d:9d:ad,model=virtio" \
--network="network=net8,mac=52:54:00:21:f8:29,model=virtio" \
--noautoconsole --graphics none --dry-run --controller="type=usb,model=none" --print-xml \
--console "pty,target_type=virtio" > vm.xml

virsh define vm.xml
virsh start vm

Boot VM parameter

vga=791 boot=live dhcp lvm noprompt BOOTIF=01-52-54-00-21-f8-29 hostname=vm random.trust_cpu=on noeject

VM dependencies

apt install build-essential autotools-dev libdumbnet-dev libluajit-5.1-dev \
libpcap-dev libpcre3-dev zlib1g-dev pkg-config libhwloc-dev cmake \
liblzma-dev openssl libssl-dev cpputest libsqlite3-dev libtool git \
autoconf bison flex asciidoc source-highlight intel-cmt-cat ethtool

Setup Snort on DuT

# prepare link interfaces
for port in ens3 ens4; do
	ip link set dev $port up
	ip link set dev $port promisc on
	ip link set dev $port arp off
	ethtool -K $port gro off lro off
	ethtool -K $port sg off tso off gso off
	sysctl -w net.ipv6.conf.$port.disable_ipv6=1
done

# change outgoing mac to enable forwarding for SR-IOV
arptables -A OUTPUT -o ens5 --h-length 6 -j mangle --mangle-mac-d '00:11:22:33:44:55'

# download snort configs
git clone https://github.com/gallenmu/low-latency /root/low-latency

# build daq
wget -q https://www.snort.org/downloads/snortplus/daq-2.2.2.tar.gz
tar -xzf daq-2.2.2.tar.gz
cd ~/daq-2.2.2
./configure --disable-dump-module --disable-ipfw-module --disable-ipq-module --disable-nfq-module --disable-pcap-module --disable-netmap-module --enable-static --disable-shared --prefix=/opt/snort
make
make install
ldconfig

# build snort
export PATH=$PATH:/opt/snort/bin/
cd /root
wget -q https://www.snort.org/downloads/snortplus/snort-3.0.0-beta.tar.gz
tar -xzf snort-3.0.0-beta.tar.gz
cd /root/snort-3.0.0
./configure_cmake.sh --prefix=/opt/snort --with-daq-libraries=/opt/snort/lib --with-daq-includes=/opt/snort/include --enable-static-daq --disable-docs
cd build
make -j 4
make install

Setup timestamper

git clone https://github.com/gallenmu/MoonGen /root/moongen
cd /root/moongen
git checkout 0450d0f84ac900843031b61a9f28a8194add147d 
/root/moongen/build.sh
/root/moongen/bind-interfaces.sh
/root/moongen/setup-hugetlbfs.sh

Experiment execution

1. Start Snort forwarder in the VM running on the DuT:

Start Snort

taskset -c 1 /opt/snort/bin/snort -c /root/low-latency/configs/snort/snort-norules.lua --daq-dir /usr/local/lib/daq --daq-dir /opt/snort/lib/daq --daq afpacket --daq-var fanout_type=hash --max-packet-threads 1 -Q -i ens3:ens4

2. Wait until Snort is started, then start MoonGen on the LoadGen:

Start MoonGen

/root/moongen/build/MoonGen /root/moongen/examples/moonsniff/traffic-gen.lua --src-mac 52:54:00:8d:9d:ad --dst-mac 52:54:00:21:f8:29 --fix-packetrate 10000 --packets 300000 --warm-up 30 --l4-dst 53 --flows 10 --burst 1 1 0

3. After Moongen on the LoadGen has started 1000 packets are sent to warm up the DuT. After that packet generation is paused for 30 seconds. In this pause MoonGen's packet sniffer on the timestamper should be started to record the actual measurement:

Start MoonGen

/root/moongen/build/MoonGen /root/moongen/examples/moonsniff/sniffer.lua 1 0 --capture --time 60 --snaplen 256

4. The timestamper stops recording after 60 seconds and creates the two pcap files latencies-pre.pcap and latencies-post.pcap which can be used for evaluation.

Steps to reproduce figures

Raw data

Pcaps

Evaluation execution

Setup evaluator

# install dependencies
apt install postgresql postgresql-client parallel python3-pip texlive-full zstd
python3 -m pip install pypacker

Generate figures

git clone https://github.com/gallenmu/low-latency

#decompress pcaps
cd low-latency/measurements
for i in $(find . -iname *.zst); do zstd --rm -d $i; done;

#create figures
cd ../figures/motivation/pcap
su postgres -c "createuser -s root"
dropdb --if-exists root
createdb root
export PGDATABASE=root
../../../scripts/pcap-import.sh ../../../measurements/motivation/pcap/latencies-pre-rate10000-snort-norules.pcap
../../../scripts/pcap-analysis.sh ../../../measurements/motivation/pcap/latencies-pre-rate10000-snort-norules.pcap
python3 ../../../scripts/pcap-tex-create.py .
for i in *.tex; do lualatex $i; done

Output

The precompiled output files can be found in the repository. For this measurement all-filtered.pdf and latencies-pre-rate10000-snort-norules.pcap.worst-filtered.pdf were used.