IPVS Using IPIP Tunnel

General Setup

In this introduction, we use Virtualbox as our VM environment.

Here are the three servers we need:

• LVS server : 2CPU core, 2G Ram, Host network vboxnet 2, vboxnet 3
• Service server : 1CPU core, 1G Ram, Host network vboxnet 2, vboxnet 3
• Test server: 1CPU core, 1G Ram, Host network vboxnet 2, vboxnet 3

Host network setting in virtualbox

IPIP Tunnel Setup

LVS server

Let’s collect our network info from ifconfig

enp0s8: 192.168.58.3
enp0s9: 192.168.59.3

Setup virtual interface enp0s8:0 for IPIP tunnel

VIP: 192.168.58.100

sudo ifconfig enp0s8:0
# the same subnet in enp0s8
# VIP 
sudo ifconfig enp0s8:0 192.168.58.100 netmask 255.255.255.255 broadcast 192.168.58.100
# enable ip forwarding
sudo echo 1 > /proc/sys/net/ipv4/ip_forward
# setup 10.0.2.100 in ipvsadm
# -s (scheduler) wlc,rr,....
sudo ipvsadm -A -t 192.168.58.100:8000 -s rr
# -t for TCP, can use -d for UDP
# -r remote <IP>:<Port> for IPIP tunnel here we choose 23 (telnet)
# -i specify IPIP tunnel
sudo ipvsadm -a -t 192.168.58.100:8000 -r 192.168.58.4 -i

Service Server

Ifconfig Info

enp0s8: 192.168.58.4
enp0s9: 192.168.59.4

Setup tunnel interface

# enable ip forwarding
sudo echo 1 > /proc/sys/net/ipv4/ip_forward
# arp settings
sudo sysctl net.ipv4.conf.all.arp_ignore=1
sudo sysctl net.ipv4.conf.all.arp_announce=2
sudo sysctl net.ipv4.conf.tunl0.arp_ignore=1
sudo sysctl net.ipv4.conf.tunl0.arp_announce=2
# loose mode on rp_filter (reverse packet filtering)
# allow packets came from other interface (src ip) 
sudo sysctl net.ipv4.conf.tunl0.rp_filter=2
# tunnel interface
sudo ifconfig tunl0 192.168.58.100 netmask 255.255.255.255 broadcast 
192.168.58.100 up
# may not need it if it is already in route table
sudo route add -net 192.168.58.0 netmask 255.255.255.0 dev enp0s8
# add IPVS server ip to route table
sudo route add -host 192.168.58.100 dev tunl0

Testing

Use a simple python http server to test how IPVS works

Service server

# run simple http server which will list files in current folder
python3 -m http.server 8000

Test server

Network interface

enp0s8: 192.168.58.5
enp0s9: 192.168.59.5

Curl the port 8000

# access the API via virtual IP
curl http://192.168.58.100:8000

Capture Packets

We can capture the packets by tcpdump

# check the packet from port 8000
# -i : interface
# sniffer from LVS server to see the packets to port 8000
tcpdump -i enp0s8 port 8000

Reference

Virtual Server via IP Tunneling

Understanding the rp_filter modes and log_martians in Linux

Reverse Path Filtering

An introduction to Linux virtual interfaces: Tunnels | Red Hat Developer