In Linux, network-based applications rely on the kernel’s networking stack to establish communication with other systems. While this process is generally efficient and has been optimized over the years, in some cases it can create unnecessary overhead that can impact the overall performance of the system for network-intensive workloads such as web servers and databases.

XDP (eXpress Data Path) is an eBPF-based high-performance datapath inside the Linux kernel that allows you to bypass the kernel’s networking stack and directly handle packets at the network driver level. XDP can achieve this by executing a custom program to handle packets as they are received by the kernel. This can greatly reduce overhead, improve overall system performance, and improve network-based applications by shortcutting the normal networking path of ordinary traffic. However, using raw XDP can be challenging due to its programming complexity and the high learning curve involved. Solutions like Calico Open Source offer an easier way to tame these technologies.

Calico Open Source is a networking and security solution that seamlessly integrates with Kubernetes and other cloud orchestration platforms. While infamous for its policy engine and security capabilities, there are many other features that can be used in an environment by installing Calico. These include routing, IP address management, and a pluggable data plane with various options such as eBPF, IPtables, and VPP. In addition to these features, Calico also makes it simple to create and load custom XDP programs to target your cluster host interfaces.

The power of XDP can be used to improve the performance of your host workloads by using the same familiar Kubernetes-supported syntax that you use to manage your cluster resources every day. Calico’s integration with XDP works and is implemented in the same way for any cluster running Linux and Calico, whether it uses IPtables, IP Virtual Server (IPVS), or Calico’s eBPF data plane.

By the end of this article, you will know how to harness the power of XDP programs to accelerate your host workloads without needing to learn any programming languages. You will also learn more about these technologies, how Calico offers an easier way to adapt to the ever-changing cloud computing scene, and how to use these cutting-edge technologies to boost your cluster performance.

XDP workload acceleration

One of the main advantages of XDP for high-connection workloads is its ability to operate at line rate with low overhead for the system. Because XDP is implemented directly in the kernel at the earliest possible execution point, it can process packets at very high speeds with minimal latency. This makes it well-suited for high-performance networking applications that require fast and efficient packet processing.

The following image illustrates the packet flow in the Linux kernel.

To better understand the advantages of XDP, let’s examine it in a common setup by running an in-memory-database on a Kubernetes host and using XDP to bypass the Linux conntrack features.

Host network security

A host-based workload is a type of application that runs directly on the host machine. It is often used to describe workloads that are not deployed in a container orchestration platform like Kubernetes, but rather directly on a host.

The following code block illustrates a host workload network socket.

# ss -lnatp | egrep "\*:6443"
LISTEN    0      4096                         *:6443                        *:*     users:(("kube-apiserver",pid=609,fd=7)) 

This type of workload is quite difficult to secure by using the ordinary Kubernetes network policy (KNP) resource, since host workloads do not belong to any of the namespaces that Kubernetes moderates. In fact, one of the shortcomings of KNP is its limitation in addressing these types of traffic. But don’t worry, the modular nature of Kubernetes allows us to use Container Networking Interface (CNI) plugins, such as Calico, to address such issues.

Calico Host Endpoint Policy (HEP)

Calico host endpoint policy (HEP) is a Kubernetes custom resource definition that enables the manipulation of host traffic within a cluster. A HEP in Calico represents a host participating in a cluster and the traffic generated by the workload on that host. HEPs can be associated with host network cards, and allow you to apply network security policies to the traffic generated by the workload on the host.

A host endpoint policy is defined using the HostEndpoint resource in Calico, and it has the following structure:

apiVersion: projectcalico.org/v3
kind: HostEndpoint
metadata:
 name: node2-ens4
 labels:
   hep: redis-host
spec:
 interfaceName: ens4
 node: node2
 expectedIPs:
   - <NODE_IP>

The metadata field contains information about the host endpoint, such as its name and any labels that are associated with it, similar to other Kubernetes resources these labels can be used with other resources to reference the traffic to or from a HEP.

The spec field contains the configuration for the HEP, including the name of the interface that it is associated with, the node on which it is running, and the expected IP addresses of the designated Kubernetes node network interface card.

Using HostEndpoint with security policies

Similar to other Kubernetes security resources, HEP will become a deny-all behavior and impose a lockdown on your cluster in the absence of an explicit permit that requires a preemptive look at the traffic that should be allowed in your cluster before implementing such a resource. But on top of its security advantages, you can refer to HEP labels from other Calico security resources such as global security policies to control traffic that might otherwise be difficult to control, and create more complex and fine-grained security rules for your cluster environment.

The following selector in a security policy would allow you to filter packets that are associated with the redis-host host endpoint policy:

selector: has(hep) && hep == "redis-host"

This selector matches packets that have the hep label with a value of redis-host. You can use this selector in combination with other rules in a security policy to specify how the matched packets should be treated by your CNI. In the next section, we will use the same logic to bypass the Linux conntrack feature.

Connection tracking in Linux

By default, networking in Linux is stateless, meaning that each incoming and outgoing traffic flow must be specified before it can be processed by the system networking stack. While this provides a strong security measure, it can also add complexity in certain situations. To address this, conntrack was developed.

Conntrack, or connection tracking, is a core feature of the Linux kernel used by technologies such as stateful firewalls. It allows the kernel to keep track of all logical network connections or flows by maintaining a list of the state of each connection, such as the source and destination IP addresses, protocol, and port numbers. This list has an adjustable soft limit, meaning that it can expand as needed to accommodate new connections. However, in some cases, such as with short-lived connections, conntrack can become a bottleneck for the system and impact performance.

For example, this is the limit on my local computer.

 cat /proc/sys/net/netfilter/nf_conntrack_max 
262144

While it is possible to dig deeper into the conntrack table details from the `/proc/sys/net/netfilter` path in your system, applications such as conntrack can give you a more organized view of these records.

The following code block illustrates recorded entries by my local Kubernetes host.

# conntrack -L
tcp      6 431979 ESTABLISHED src=172.18.0.3 dst=172.18.0.4 sport=53522 dport=6443 src=172.18.0.4 dst=172.18.0.3 sport=6443 dport=53522 [ASSURED] mark=0 use=1
tcp      6 6 CLOSE src=127.0.0.1 dst=127.0.0.1 sport=44888 dport=10257 src=127.0.0.1 dst=127.0.0.1 sport=10257 dport=44888 [ASSURED] mark=0 use=1

In addition to securing a cluster, a host endpoint selector can be added to a Calico global security policy with a `doNotTrack` value to bypass Linux tracker capabilities for specific flows. This method could be beneficial for network-intensive workloads that receive a massive number of short-lived requests and prevent the Linux conntrack table from overflowing.

The following code block is a policy example that bypasses the Linux conntrack table.

piVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
 name: redis-accelerator-policy
spec:
 selector: has(hep) && hep == "redis-host"
 order: 9
 applyOnForward: true
 doNotTrack: true
 ingress:
   - action: Allow
     protocol: TCP
     destination:
       ports:
         - 6379
 egress:
   - action: Allow
     protocol: TCP
     source:
       ports:
         - 6379

It is worth noting that since `doNotTrack` disables Linux conntrack capabilities, any traffic that matches the previous policy will become a stateless connection. In order to return it to its originating source, we have to specifically add a return policy (egress) to our Calico global security policy resource.

Calico’s workload acceleration is not limited to XDP. In fact, Calico’s eBPF data plane can be used to provide acceleration for other types of workloads in a cluster. If you would like to learn more about Calico’s eBPF data plane, please click here.

Conclusion

Overall, eBPF and XDP are powerful technologies that offer significant advantages for high-connection workloads, including high performance, low overhead, and programmability. In this article, we established how Calico makes it easier to take advantage of these technologies without worrying about the complexities associated with the learning curve involved in learning these complicated technologies.

Check out our free self-paced workshop, Turbocharging host workloads with Calico eBPF and XDP, to learn more.