The following tweaks can be applied to Intel based NIC's with the relevant features. Some configuration applies to the system in general and not just to Intel NIC's.
Check if the interface is dropping RX packets due to the buffers being full:
ethtool -S eth0 | grep "rx_dropped"
If there are drops, check the number of descriptors assigned using ethtool -g:
# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 512
RX Mini: 0
RX Jumbo: 0
TX: 512
From the above output the RX buffer can be raised up to 4096:
ethtool -G eth0 rx 4096
aRFS is a load-balancing mechanism that allows you to direct packets to the same CPU where an application is running or consuming the packets in that flow. To use aRFS there are certain caveats:
To setup aRFS:
INTERFACE=eth1
ethtool -K $INTERFACE ntuple on
NUM_RPS_ENTRIES=16384
NUM_RX_QUEUES=64
echo $NUM_RPS_ENTRIES > /proc/sys/net/core/rps_sock_flow_entries
for file in /sys/class/net/$INTERFACE/queues/rx-*/rps_flow_cnt; do
echo $(($NUM_RPS_ENTRIES/$NUM_RX_QUEUES)) > $file;
done
systemctl stop irqbalance
Configure interrupt affinity.
Set the server application on the desired CPU.
Note: The above configuration will be lost on reboot.