トンネリングプロトコルのパフォーマンス比較
May 15, 2021
TL;DR
トンネルなし > ip6ip6 = ip6gre = SRv6 > WireGuard
はじめに
自分の中で ip6ip6 トンネルが謎に遅いイメージ(*1)があったため、実際に比較してみることにした
比較対象
- nexthop 指定 (トンネルなし)
- ip6ip6
- ip6gre
- SRv6
- (おまけ) WireGuard
- 暗号化・非暗号化のプロトコルが混ざっていて適切ではないが、home use なので厳密に分類しない。
- IPSec hw offload 対応の NIC が入手できたら追加計測したい。
構成
追記
- vcpu: 1
- memory: 2G
- nic: VirtIO
- OS: ubuntu 20.04.2
- kernel: 4.0-73-generic
- wireguard: 1.0.20200513
rss 等を考慮したくないため、vcpu が１つの環境で検証を行った。
オフロードに関しては付録を参照。
結果
| mtu | 1280 byte | 512 byte | 256 byte |
|---|---|---|---|
| トンネルなし | 6.97 Gbps | 2.59 Gbps | 1.18 Gbps |
| ip6ip6 | 4.56 Gbps | 1.66 Gbps | 0.66 Gbps |
| ip6gre | 4.58 Gbps | 1.62 Gbps | 0.65 Gbps |
| srv6 | 4.54 Gbps | 1.63 Gbps | 0.66 Gbps |
| wireguard | 0.98 Gbps | 0.54 Gbps | 0.29 Gbps |
(見えにくいが ip6ip6、ip6gre、srv6 の線が重なっている。)
感想
非常に妥当な結果になった。
しかし、思っていた以上に SRv6 が早くてびっくりした。複雑な処理を行ってそう（根拠不足）なのでもう１周りくらい遅いと思っていた。
ip6ip6 が謎に遅いと思っていたのは気のせいだった。
実際の Configuration
ホスト A の設定
sysctl -w net.ipv6.conf.all.forwarding=1
sysctl -w net.ipv6.conf.all.seg6_enabled=1
ip netns add h
ip netns exec h ip link set lo up
ip link add h1 type veth peer name h2
ip link set h1 netns h
ip link set h2 up
ip netns exec h ip link set h1 up
ip addr add fc00::0/127 dev h2
ip netns exec h ip addr add fc00::1/127 dev h1
ip netns exec h ip route add ::/0 via fc00::0
ip netns exec h ip addr add fc00::1:0 dev lo
ip netns exec h ip addr add fc00::1:1 dev lo
ip netns exec h ip addr add fc00::1:2 dev lo
ip netns exec h ip addr add fc00::1:3 dev lo
ip netns exec h ip addr add fc00::1:4 dev lo
ip route add fc00::1:0/112 via fc00::1
ip route add fc00::12:0/112 via [ホストBアドレス] dev ens18
# トンネルなし
ip route add fc00::2:0/128 via [ホストBアドレス]
# ip6ip6
ip addr add fc00::11:1 dev lo
ip tunnel add tun0 mode ip6ip6 remote fc00::12:1 local fc00::11:1
ip link set tun0 up
ip route add fc00::2:1/128 dev tun0
# ip6gre
ip addr add fc00::11:2 dev lo
ip tunnel add tun1 mode ip6gre remote fc00::12:2 local fc00::11:2
ip link set tun1 up
ip route add fc00::2:2/128 dev tun1
# srv6
ip route add fc00::2:3/128 encap seg6 mode encap segs fc00::12:3 dev ens18
ip route add fc00::11:3/128 encap seg6local action End.DX6 nh6 fc00::1 dev h2
# wireguard
ip addr add fc00::11:4 dev lo
ip link add dev wg0 type wireguard
wg set wg0 listen-port 51820 private-key /etc/wireguard/private.key
wg set wg0 peer lgHivh3G2hQFEDESTfk5mhiz6suArWxhLniUxH2XLRM= endpoint "[fc00::12:4]:51820" allowed-ips ::/0
ip link set wg0 up
ip route add fc00::2:4/128 dev wg0
ホスト B の設定
sysctl -w net.ipv6.conf.all.forwarding=1
sysctl -w net.ipv6.conf.all.seg6_enabled=1
ip netns add h
ip netns exec h ip link set lo up
ip link add h1 type veth peer name h2
ip link set h1 netns h
ip link set h2 up
ip netns exec h ip link set h1 up
ip addr add fc00::2/127 dev h2
ip netns exec h ip addr add fc00::3/127 dev h1
ip netns exec h ip route add ::/0 via fc00::2
ip netns exec h ip addr add fc00::2:0 dev lo
ip netns exec h ip addr add fc00::2:1 dev lo
ip netns exec h ip addr add fc00::2:2 dev lo
ip netns exec h ip addr add fc00::2:3 dev lo
ip netns exec h ip addr add fc00::2:4 dev lo
ip route add fc00::2:0/112 via fc00::3
ip route add fc00::11:0/112 via [ホストAアドレス] dev ens18
# トンネルなし
ip route add fc00::1:0/128 via [ホストAアドレス]
# ip6ip6
ip addr add fc00::12:1 dev lo
ip tunnel add tun0 mode ip6ip6 remote fc00::11:1 local fc00::12:1
ip link set tun0 up
ip route add fc00::1:1/128 dev tun0
# ip6gre
ip addr add fc00::12:2 dev lo
ip tunnel add tun1 mode ip6gre remote fc00::11:2 local fc00::12:2
ip link set tun1 up
ip route add fc00::1:2/128 dev tun1
# srv6
ip route add fc00::1:3/128 encap seg6 mode encap segs fc00::11:3 dev ens18
ip route add fc00::12:3/128 encap seg6local action End.DX6 nh6 fc00::3 dev h2
# wireguard
ip addr add fc00::12:4 dev lo
ip link add dev wg0 type wireguard
wg set wg0 listen-port 51820 private-key /etc/wireguard/private.key
wg set wg0 peer fltimlxR8KXkESi+LlpVU2QmVKsuMPl0im3hnVOp1jc= endpoint "[fc00::11:4]:51820" allowed-ips ::/0
ip link set wg0 up
ip route add fc00::1:4/128 dev wg0
付録
オフロード
# ethtool -k ens18
Features for ens18:
rx-checksumming: on [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: on [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]