Policy-based routing with Egress Gateways

In this blog post I will be discussing how Egress Gateways can help in securing traffic by means of policy routing. If you are not familiar with Egress Gateways, take a look at our blog post, Using Calico egress gateway and access controls to secure traffic.

Challenges

Modern applications built on microservices architecture need to communicate with external services, however, the context of these connections (source IPs) is not maintained when traffic leaves the Kubernetes cluster, creating several challenges. Let’s explore those through different scenarios:

Integration with legacy firewalls

One of the most common requirements is to integrate a Kubernetes cluster with perimeter based legacy firewalls that cannot use native Kubernetes metadata as labels to enforce security controls on the traffic. Although Calico policies would cover such scenarios, there are several reasons companies will need to maintain this second layer of security.

As legacy firewalls use IP addresses to identify endpoints, and in a Kubernetes environment, IPs are ephemeral in nature, the network security person needs to implement workarounds to preserve them. Instead, Calico can help in following two ways:

• Using its advanced IPAM engine to assign specific IP addresses to certain deployments, and BGP to expose those to the firewall device through the infrastructure, so overlays or NAT are not used.
• Using Egress Gateways to fix those IP addresses as depicted in Figure 1:

Figure 1

The first approach may require a lot of planning, as pools must be sized accordingly, and the proper operation of the application may require coordination among different teams working together. Let’s assume we have two different teams working on the same platform: the developers who must include the proper annotation on their containerized application, so they get the proper IP addresses, and the infrastructure team, which controls the IP pools, and implements the security controls. If there is a misalignment with IP management, applications can fail to be deployed correctly.

With Egress Gateways, the entire IP management resides within a single team (i.e. the platform team), so even when a misconfiguration still could prevent workloads intended to use an egress gateway from reaching external destinations as designed, there will be no need of coordination to get IPs from specific IP pools, and the IP address management will be greatly simplified.

Although this use case has been traditionally supported by the Egress Gateway feature in Calico, there are other scenarios where most complex topologies must be supported. I will discussing those in the following two sections:

Access to external managed services

Many SaaS or external facing applications need to connect their applications to external controlled services that will accept traffic from specific IP addresses only.

So, as depicted in Figure 2, Pod ‘B’ must access two different external services which expect a specific IP address to connect to them. Policy-based routing allows us to cover this scenario, as traffic will be redirected to each service as described in the section “Policy-Based routing for Egress-Gateway”.

Figure 2

Single point of exit from Kubernetes cluster

Users sometimes need to control the point where traffic leaves their Kubernetes cluster. This can be due to several reasons, but the main one is based on security concerns.

There are several ramifications on this approach, and Figure 3 illustrates this:

Figure 3

The pods who have been provided external communication must exist through Node X in the diagram, as that is one of the subset of nodes which has connectivity to the VLAN where access to the public internet is made, while other Kubernetes nodes may not have access to it. This physical isolation provides the ability to implement other security measures as external physical Intrusion Prevention systems to better control this traffic. (Note: Calico supports IDS features at the workload level that can complement those.)

We can implement an Egress Gateway deployment that gets scheduled in that node for this purpose. But, what happens if those pods with public internet access need to access some internal resources that cannot be routed through that DMZ VLAN as depicted in the figure, but still need to be accessed from specific IP addresses?. This is another scenario where Policy-Based routing for Egress Gateways comes to the rescue!

Policy-Based routing for Egress-Gateway

We have discussed different situations in which we need to have control over both, the source IP address of our traffic (used as an identifier for legacy infrastructure), and the ability to make decisions based on the destination of that traffic.

But what does change in terms of implementation when doing Policy-Based routing for Egress Gateways? There is a new resource in the latest Calico releases called EgressGatewayPolicy. Let’s take a look at how this resource would look like:

apiVersion: projectcalico.org/v3
kind: EgressGatewayPolicy
metadata:
  name: "egw-policy"
spec:
  rules:
  - destination:
      cidr: 172.16.1.0/24
    description: "Local: no gateway"
  - destination:
      cidr: 192.0.2.0/24
    description: "Gateway to red network"
    gateway:
      namespaceSelector: "projectcalico.org/name == 'default'"
      selector: "egress-code == 'red'"
  - destination:
      cidr: 198.51.100.0/24
    description: "Gateway to blue network"
    gateway:
      namespaceSelector: "projectcalico.org/name == 'default'"
      selector: "egress-code == 'blue'"

As noted in the manifest, the pod will use a different egress gateway based on the destination subnet of the service, and additionally, we will be able to present a different source IP based on that criteria. If the service uses an allowlist of IP addresses, we will be able to meet their enforcement requirements no matter what dynamic IP the source container gets.

But how does Calico indicate to the workload it must use this policy resource?. This is done by means of an annotation in the namespace. Taking as example namespace “app1”:

kubectl annotate ns app1 egress.projectcalico.org/egressGatewayPolicy="egw-policy"

For further details around this feature visit our related documentation.

Summary

Policy-based routing for egress gateways provides users the required flexibility to implement security controls in more advanced scenarios, where tying containers to a single egress gateway is not enough.

Ready to try Calico for yourself? Get started with a free Calico Cloud trial.