By default, traffic leaving a Kubernetes cluster lacks a meaningful network identity, making it challenging to associate it with its source workload. This is an issue because, in an on-premises infrastructure, companies rely on firewalls, for example Sophos Firewall, to inspect this traffic which loses its identity as soon as it leaves the cluster. Sophos Firewall provides the perimeter-level security for clusters but lacks workload level security because Kubernetes is inherently dynamic resulting in the use of new IP addresses or traffic appearing to originate from cluster node IPs that also scale dynamically.

Egress Gateways through Calico offer a solution by enabling the identification of Kubernetes namespaces and pods linked to egress traffic from clusters. This simplifies the identification of the source of traffic for inspection by Sophos Firewall and for security, troubleshooting or management purposes.

Requirements:

• Calico Cloud or Enterprise
• Calico CNI
• Sophos Firewall

We have seen several inquiries from administrators who need to filter and control traffic from a Kubernetes cluster to their internal servers or to the public internet.

In this blog, I will show how you can leverage Egress Gateway with Calico to identify traffic from specific namespaces and filter them with Sophos Firewall, according to your needs.

BGP CONFIGURATION

For both scenarios, we need to allow communication between the cluster and Sophos Firewall by using BGP. Here are the 2 guides to setup BGP:

• BGP configuration on Sophos Firewall
• BGP configuration on Calico Cloud

For these scenarios, I have used a BGP node-to-node mesh and peered Sophos Firewall with all nodes.

Node details:

• Control: 172.17.17.51
• Worker1: 172.17.17.52
• Worker2: 172.17.17.53

Sophos Firewall LAN IP (on Port1): 172.17.17.210

BGP peers from Control:

kubectl get caliconodestatus my-caliconodestatus-1 -o jsonpath="{range .status.bgp.peersV4[*]}{}{'\n'}{end}"
{"peerIP":"172.17.17.52","since":"13:40:21","state":"Established","type":"NodeMesh"}
{"peerIP":"172.17.17.53","since":"13:40:04","state":"Established","type":"NodeMesh"}
{"peerIP":"172.17.17.210","since":"13:40:02","state":"Established","type":"GlobalPeer"}

BGP peers from Sophos Firewall:

bgp> show bgp summary 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
172.17.17.51    4      64567       251       205        0    0    0 01:35:03            8        1 N/A
172.17.17.52    4      64567       250       200        0    0    0 01:34:43            8        1 N/A
172.17.17.53    4      64567       265       202        0    0    0 01:35:01            9        1 N/A

TRAFFIC WITHOUT EGRESS GATEWAY

At this stage, the traffic leaving the cluster will be SNATed with the node IPs and, as per configuration on the firewall, packets sent to the DMZ server (172.19.19.50) are denied.

Details about the test:

• DMZ server: 172.19.19.50
• App used: multitool (multi-red in red namespace & multi-blue in blue namespace)
• Both apps are currently running on the same node worker2
• Worker2 IP: 172.17.17.53
• Firewall rules in Sophos Firewall are not allowing traffic from the whole LAN network (172.17.17.0/24) but only from the egress-gateway-red cidr (172.17.17.32/31) which we will create later
• I’ve tested ICMP and HTTP connection

From the screenshot below, you can see that packets are SNATed with the worker1 IP (nodeIP) and dropped by the firewall. Packets are leaving from different pods (multi-blue & multi-red) but it’s impossible for the Sophos Firewall to tell the difference.

FROM THE CLUSTER:

FROM THE SOPHOS FIREWALL:

TRAFFIC WITH 2 EGRESS GATEWAYS

I’ll set up 2 Egress Gateways and assign each one to the related namespace:

• Egress-gateway-red to red namespace
• Egress-gateway-blue to blue namespace

The Calico Cloud Egress Gateway has been enabled for Namespace:

kubectl patch felixconfiguration default --type='merge' -p \
    '{"spec":{"egressIPSupport":"EnabledPerNamespace"}}'

Each Egress Gateway will have 2 replicas and 2 possible IP addresses:

• Egress-gateway-red: cidr 172.17.17.32/31 (172.17.17.32 – 172.17.17.33)
• Egress-gateway-blue: cidr 172.17.17.34/31 (172.17.17.34 – 172.17.17.35)

Two Egress Gateway policies have been created, one per each Egress Gateway. Here the full guide about Egress Gateway: Configure egress gateways, on-premises

Namespaces blue and red have been annotated to apply the right gateways and policies to each of them.

On Sophos Firewalls, only cidr 172.17.17.32/31 (172.17.17.32 – 172.17.17.33) can send ICMP packets or establish an HTTP connection. In fact, from these screenshots, we see that only this traffic is allowed:

FROM THE CLUSTER:

FROM THE SOPHOS FIREWALL:

As platform, network security and devops teams are now responsible for bringing security at workload level and have to manage it with Sophos firewalls, they require network identity for these workloads that extends beyond their clusters. With this blog, I have provided a framework for how to integrate Egress Gateway with Calico, with Sophos Firewall to extend your workload’s identity to Sophos firewall. If you have questions, feel free to contact us here.

Ready to get started? Try Calico Cloud for free.