This page contains WireGuard benchmark information.
These tools are using for testing throughput and latency:
- irtt - Latency/packetloss measurement
- iperf2 - Bandwidth measurement for TCP and UDP with varying packet sizes; used due to thread support
- iperf3 - Bandwidth measurement for TCP and UDP with varying packet sizes; used for single thread throughput
- tc - Used to simulate latency and packet loss to immitate real world conditions of VPN clients
For reporting purposes these tools are used:
- bwm-ng - Average throughput speed/packets
- netdata - Various high resolution metrics
The following VPN implementations will be used for comparisons:
Two servers are used; a source server (used to initiate the tests and act as the VPN client) and a target server (used to run the various daemons and act as the VPN server).
iperf2 has a daemon bound to each NIC used for TCP testing with the following command line:
iperf -D -p 3002 -s -B <NIC 1 IP>
iperf -D -p 3002 -s -B <NIC N IP>
iperf2 has a daemon bound to each NIC used for UDP testing with the following command line:
iperf -D -u -p 3001 -s -B <NIC 1 IP>
iperf -D -u -p 3001 -s -B <NIC N IP>
iperf3 has a daemon bound to each NIC used for TCP and UDP testing with the following command line:
iperf3 -D -s -B <NIC 1 IP>
iperf3 -D -s -B <NIC N IP>
irtt is a systemd managed service in server mode (executed with the command line /usr/bin/irtt server).
An OpenVPN server instance is used per NIC being tested.
config to be put here
config to be put here
config to be put here
The following tests will be executed; tests will be executed in parallel for each NIC available (eg. if two NIC's are being used for testing, 2 tests will be executed in parallel to use both NIC's at the same time):
- iperf3:
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b RX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b TX.
- iperf2:
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 2 parallel threads RX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 4 parallel threads RX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 8 parallel threads RX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 2 parallel threads TX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 4 parallel threads TX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 8 parallel threads TX.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 2 parallel threads in duplex mode.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 4 parallel threads in duplex mode.
- TCP and UDP packet size of 1500b, 1250b, 1000b, 750b, 500b, 250b and 64b with 8 parallel threads in duplex mode.
The tests will exeuted for 5 minutes and be repeated for each of these cases:
- Base test with no VPN
- OpenVPN
- strongSwan
- WireGuard
For each test case, they will be repeated with the following variables:
- Latency of 25ms, 50ms, 100ms, 175ms and 250ms.
- Packet loss of 1%, 5% and 10%.
Metrics will be gathered for during each test case:
- Average bandwidth
- Average packets/sec
- Average packet loss
- CPU usage
- Latency/jitter with
irtt
To allow future review of other collected metrics, a netdata snapshot will be taken for each test from both the server and client.