Get Start with KNI example
Compile DPDK
# You may need to install numa-devel package if make throws an exception.
export RTE_SDK=/home/ramichen/project/dpdk
export RTE_TARGET=build
make defconfig
make
make -C examples
Insert kernel module
modprobe uio
insmod $RTE_TARGET/kmod/rte_kni.ko
#insmod build/kmod/rte_kni.ko kthread_mode=multiple
insmod $RTE_TARGET/kmod/igb_uio.ko
Mount Hugetble pages
DPDK makes use of hugetbles as packets pool to exchange packets between kni kernel module and userspace dpdk application.
# check what's the default page size
cat /proc/meminfo | grep Huge
# if its a 2MB page size
echo 2048 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# if its a 1GB page size
echo 2 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
# mount it to a directory to be used by dpdk application
mkdir -p /mnt/huge
mount -t hugetlbfs -o pagesize=1G,size=4G nodev /mnt/huge
Rebind a new module for your NIC
usertools/dpdk-devbind.py --status | grep 0000
ip link set enp0s31f6 down
usertools/dpdk-devbind.py --bind=igb_uio 0000:00:1f.6
Start KNI example
examples/kni/build/kni -l 0-1 -n 2 -- -P -p 0x1 --config="(0,0,1)"
ip ad add 192.168.0.1/24 dev vEth0
ip link set vEth0 up
ping -c 3 192.168.0.1
# send SIGUSR1 signal to DPDK KNI app
pkill -10 kni
# examples/kni/build/kni -l 0-1 -n 2 -- -P -p 0x1 --config="(0,0,1)"
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:15b7 net_e1000_em
APP: Initialising port 0 ...
KNI: pci: 00:1f:06 8086:15b7
Checking link status
.done
Port0 Link Up - speed 100Mbps - full-duplex
APP: Lcore 1 is writing to port 0
APP: Lcore 0 is reading from port 0
APP: Configure network interface of 0 up
APP: Configure network interface of 0 up
**KNI example application statistics**
====== ============== ============ ============ ============ ============
Port Lcore(RX/TX) rx_packets rx_dropped tx_packets tx_dropped
------ -------------- ------------ ------------ ------------ ------------
0 0/ 1 17866 0 1025 942
====== ============== ============ ============ ============ ============
What’s going on
For a physical NIC port suck as Port 0, CoreA0 reads from the port and writes to KNI devices(this is the rx work), and CoreB0 reads from KNI devices and writes the data unmodified to the physical NIC port(this is the tx work).
Packets flow
This pic shows how kernel and dpdk userspace app exchange packets.
For Ingress part
On the DPDK RX side, the mbuf is allocated by the PMD in the RX thread context. This thread will enqueue the mbuf in the rx_q FIFO. The KNI thread will poll all KNI active devices for the rx_q. If an mbuf is dequeued, it will be converted to a sk_buff and sent to the net stack via netif_rx(). The dequeued mbuf must be freed, so the same pointer is sent back in the free_q FIFO.
The RX thread, in the same main loop, polls this FIFO and frees the mbuf after dequeuing it.
Q: Why it writes to KNI device without doing anything?
A: Yes, because this is just an example. You can parse the packet and does everything you want after getting it from mbuf pool.
For Egress part
For packet egress the DPDK application must first enqueue several mbufs to create an mbuf cache on the kernel side.
The packet is received from the Linux net stack, by calling the kni_net_tx() callback. The mbuf is dequeued (without waiting due the cache) and filled with data from sk_buff. The sk_buff is then freed and the mbuf sent in the tx_q FIFO.
The DPDK TX thread dequeues the mbuf and sends it to the PMD (via rte_eth_tx_burst()). It then puts the mbuf back in the cache.
implementation
//TODO
RTE Implementation
//TODO
KNI DNS Server
//TODO
Reference
https://dpdk.org/doc/guides/sample_app_ug/kernel_nic_interface.html https://dpdk.org/doc/guides/prog_guide/kernel_nic_interface.html