Getting started with EKS and Calico

Cloud-native applications offer a lot of flexibility and scalability, but to leverage these advantages, we must create and deploy a suitable environment that will enable cloud-native applications to work their magic.

Managed services, self-managed services, and bare metal are three primary categories of Kubernetes deployment in a cloud environment. Our focus in this article will be on Amazon Web Service’s (AWS) managed Kubernetes service, Elastic Kubernetes Service (EKS), and capabilities that Calico Open Source adds to the EKS platform.

Managed services

A managed cluster is a quick and easy way to deploy an enterprise-grade Kubernetes cluster. In a managed cluster, mundane operations such as provisioning new nodes, upgrading the OS/Kubernetes, and scaling resources are transferred to the cloud provider, which allows you to expand your application with ease.

EKS is a managed service by AWS that offers a fault-tolerant Kubernetes control plane endpoint and automates worker node maintenance and deployment process.

Comparing popular CNI options in EKS

Most popular managed services, such as EKS, come with an official CNI that offers networking and other features for your cluster. While these CNIs are highly integrated with the underlying system, they can introduce some limitations. To remedy these limitations and unlock the full potential of a cloud-native environment, we can leverage the CNI chaining capabilities of CNI specification and offload some of the responsibilities to other CNIs.

In EKS, AWS-CNI provides security for a managed cluster by relying on AWS Security Groups (SG) and Kubernetes Network Policy (KNP) resources. SGs can implement security for traffic outside your cluster (traffic from VPC to Pod or vice versa), and KNP can be used to fill the gap when SGs fall short of controlling the internal communications in a cluster.

For example, SGs cannot affect the communication between two pods in the same node or pods to services. Another important fact is that applying SGs to existing pods will require a Pod restart for the change to take effect. KNPs cannot log the network security events or explicitly deny a flow. (a full list of KNP limitations)

Calico Open Source is a networking and security solution that seamlessly integrates with Kubernetes and other cloud orchestration platforms. The Calico policy engine extends the KNP concept and provides a richer set of policy capabilities. Calico has a syntax similar to KNP, which makes it easy to adopt and works in any cloud provider, i.e., you don’t need to rewrite your policies in a hybrid setup. Calico is officially introduced by Amazon as an alternative way to secure your cluster.

Calico Security Policies AWS-CNI (Pod SG + KNP)
Target Layer 3-4 of OSI Model Yes Yes
Policy priority (order) Yes Partially, SGs can have priority and order.
Ability to prevent loopback or incoming host traffic Yes Partially, SGs can prevent incoming traffic to a host.
Policy for future workloads Yes No, SG can be applied to existing resources, and KNPs need namespaces to exist in order to work.
Ability to log security events inside the cluster (Dropped flows) Yes No
Global policy that can be applied to all namespaces Yes No
Node selectors Yes No, both SG and KNP can use CIDR to reference participating Nodes.

But wait, there is much more that you can achieve with Calico!

Cloud provider IP addresses are scarce resources, and depending on your cluster size, you are entitled to a limited number of IPs, a general policy that all cloud providers share. Calico CNI IP Address Management allows you to stretch these resources as much as possible by providing private networks within your cluster.

Calico’s modular architecture also brings the power of other open-source projects to your cluster. Bird BGP routing integration allows you to extend your cluster networking with other BGP-capable devices and run a hybrid cluster. WireGuard integration of Calico provides hassle-free traffic encryption at the node and pod level that can be controlled with one command. Istio provides service mesh and application layer security for Calico open source.

Calico also offers an eBPF-based dataplane that can minimize resource utilization in your managed cluster, reduces first packet latency for packets targeting cluster services, and provide features such as source IP preservation. Since the Calico eBPF dataplane implements its routing, you can safely disable kube-proxy and take out the Network Address Translation overhead from your cluster.

Getting started with Calico on EKS

Before starting, ensure you have downloaded and configured the necessary prerequisites.

Use the following command to create a cluster:

eksctl create cluster --name my-calico-cluster --without-nodegroup

Since this cluster will use Calico for networking, you must delete the aws-node daemon set to disable AWS VPC networking for pods:

kubectl delete daemonset -n kube-system aws-node

Install the operator:

kubectl create -f https://projectcalico.docs.tigera.io/archive/v3.23/manifests/tigera-operator.yaml

Configure the Calico installation for EKS:

kubectl create -f - <<EOF

kind: Installation

apiVersion: operator.tigera.io/v1

metadata:

  name: default

spec:

  kubernetesProvider: EKS

  cni:

    type: Calico

  calicoNetwork:

    bgp: Disabled

---

apiVersion: operator.tigera.io/v1

kind: APIServer

metadata:

  name: default

spec: {}

EOF

Finally, add worker nodes to your cluster:

eksctl create nodegroup --cluster my-calico-cluster --node-type t3.medium --max-pods-per-node 100

Perfect, you are now running a managed service equipped with Calico Open Source.

Calico security policies

Now that you have a running cluster equipped with Calico Open Source, let’s deploy an application and secure it by using Calico security policies.

Use the following command to install the “wincontainer” demo application:

kubectl apply -f - <<EOF

apiVersion: v1

kind: Namespace

metadata:

  name: web-demo

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: web-container

  namespace: web-demo

  labels:

app: web-container

spec:

  replicas: 1

  selector:

    matchLabels:

      app: web-container

template:

    metadata:

      labels:

        app: web-container

    spec:

      containers:

      - name: web-container

        image: rezareza/wincontainer:latest

        imagePullPolicy: Always

        ports:

        - containerPort: 80

---

apiVersion: v1

kind: Service

metadata:

  name: container-service

  namespace: web-demo

spec:

  ports:

  - port: 80

    protocol: TCP

    targetPort: 80

selector:

    app: web-container

  type: LoadBalancer

EOF

Use the following command to determine the service address of your new workload:

kubectl get svc -n web-demo

You should see a result similar to the example below:

NAME                TYPE           CLUSTER-IP      EXTERNAL-IP                                                               PORT(S)        AGE

container-service   LoadBalancer   10.100.96.159   a9249cbcf34c94b0eaec6493a3a78b6b-1339491216.us-west-2.elb.amazonaws.com   80:32672/TCP   109s

Open up a browser and visit the service address.

Note: It may take 100s for the service DNS address to propagate.

Calico offers two types of security policies: Network Security Policy and Global Security Policy.  Similar to a KNP, Calico Network Policy can be applied to individual namespaces, while Global Security Policies have a broader reach that can extend to every corner of your cluster and underlying host resources.

Use the command below to create a global security policy:

kubectl apply -f -<<EOF

apiVersion: projectcalico.org/v3

kind: GlobalNetworkPolicy

metadata:

  name: deny-app-policy

spec:

  order: 100

  namespaceSelector: has(projectcalico.org/name) && projectcalico.org/name not in {"kube-system", "calico-system", "calico-apiserver"}

  types:

  - Ingress

  - Egress

  egress:

  - action: Allow

    protocol: UDP

    destination:

      selector: 'k8s-app == "kube-dns"'

      ports:

      - 53

EOF

Let’s head back to the browser and refresh the page.

While a Calico security policy shares a similar syntax with the Kubernetes network policy, it provides precision by allowing users to select the traffic flow to screen. For example, we have put every namespace into isolation with a single Global Security Policy, which will take multiple Kubernetes network policies to replicate. On top of that, each new namespace that we create will automatically be affected by this Global Security Policy and start in isolation, establishing networking security right from the get-go.

Note: To keep this section short, I’ve excluded “kube-system”, “calico-system”, and “calico-apiserver” namespaces from this Global Security Policy.

Now, let’s create a local policy to expose our workload to the internet. Use the following command to permit Amazon load balancer traffic to reach the workload:

kubectl apply -f -<<EOF

apiVersion: projectcalico.org/v3

kind: NetworkPolicy

metadata:

  name: service-ingress

  namespace: web-demo

spec:

  ingress:

  - action: Allow

    protocol: TCP

    destination:

      ports:

      - 80

EOF

Head back to the browser and refresh the page.

You should see a result similar to this picture:

As you can see, our containers cannot reach the Internet anymore, and we have secured our namespaced resources with just two policies.

Conclusion

Remember that this is a fraction of Calico’s capabilities and power, and there is much more to uncover. Calico policies can target interfaces, containers, pods, services, and other resources. Calico integrations can allow you to implement a zero-trust environment without breaking a sweat.

If you want to know more about EKS, eBPF, and Calico, I highly recommend that you check out our AWS and eBPF certification course.

Join our mailing list

Get updates on blog posts, workshops, certification programs, new releases, and more!