If you’re an SRE or on a DevOps team working with Kubernetes and containers, you’ve undoubtedly encountered network connectivity issues with your microservices and workloads. Something is broken and you’re under pressure to fix it, quickly. And so you begin the tedious, manual process of identifying the issue using the observability tools at your disposal…namely metrics and logs. However, there are instances where you may need to go beyond these tools to confirm a potential bug with applications running in your cluster.
Packet capture is a valuable technique for debugging microservices and application interaction in day-to-day operations and incident response. But generating pcap files to diagnose connectivity issues in Kubernetes clusters can be a frustrating exercise in a dynamic environment where hundreds, possibly thousands of pods are continually being created and destroyed.
First, you would need to identify on which node your workload is running, match your workload against its host-based interface, and then (with root access to the node) use tcpdump to generate a file for packet analysis. Then you would need to transfer the pcap files to your laptop and view them in Wireshark. If this doesn’t initially generate the information you need to identify and resolve the issue, you may have to repeat it again, potentially on another node and interface.
Calico Enterprise from Tigera introduces a new resource type called PacketCapture that automates and simplifies this cumbersome process by providing a Kubernetes-native way to capture packets from your deployments. It also provides a command-line interface to easily transfer any generated pcap files distributed across nodes directly to your local machine for analysis with tools like Wireshark.
Utilizing the same label-based selectors that are used for network policies, Calico Enterprise PacketCapture can identify a single or multiple workload endpoints for capturing live traffic. PacketCapture generates pcap files on the nodes associated with pods targeted for packet capture, and automatically manages the data retention and collection of generated files in a secure way. This significantly reduces the effort and time required to perform packet capture and analysis, and ultimately resolve connectivity issues.
Additionally, RBAC can be enabled with PacketCapture and user permissions are enforced using the standard Kubernetes RBAC based on Role and RoleBindings within a namespace. This further reduces admin overhead and makes PacketCapture an ideal solution for self-service environments and DevOps and service owners who desire greater autonomy, but require some guardrails.
Packet capture in the context of Kubernetes is time consuming. Calico Enterprise’s automated, Kubernetes-native approach significantly reduces the time and effort it takes for operators to get the network diagnostics they need to rapidly and effectively troubleshoot a connectivity issue.
Want to learn more?
Free Online Training
Access Live and On-Demand Kubernetes Training
Calico Enterprise – Free Trial
Kubernetes Networking, Security and Observability in Hybrid and Multi-Clouds