Calico on EKS Anywhere

Amazon EKS Anywhere is an official Kubernetes distribution from AWS. It’s a new deployment option for Amazon EKS that allows the creation and operation of on-premises Kubernetes clusters on your existing infrastructure.

Since its general availability release, we’ve been working hard to ensure support for Calico on EKS Anywhere, and are happy to announce that users can now choose to use Calico for container networking and security. This gives organizations already using or planning to adopt EKS Anywhere the flexibility to choose the best container networking solution for their needs. Organizations currently using Calico can add EKS Anywhere clusters and use the same Calico solution for networking and security across on-premises and cloud platforms.

Let’s take a look at how you can get started with Calico on EKS Anywhere.

Notes:

• This procedure is disruptive; it cannot be used to migrate a cluster with applications “in-service”
• Be aware that, at present, Amazon is unlikely to be able to provide support for a cluster built in this way

Install EKS Anywhere

Install EKS Anywhere as normal on vSphere, by following this documentation.

Install the Cilium CLI and uninstall Cilium

Removing Cilium from a cluster requires using the Cilium CLI, so first install it, and then use it to uninstall Cilium from the cluster:

$ curl -L --remote-name-all http://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum}
$ sha256sum --check cilium-linux-amd64.tar.gz.sha256sum
$ tar xzvf cilium-linux-amd64.tar.gz
$ rm cilium-linux-amd64.tar.gz{,.sha256sum}
$ ./cilium uninstall

Install Calico

Install Calico as normal:

$ kubectl apply -f http://docs.projectcalico.org/manifests/tigera-operator.yaml
$ kubectl apply -f http://docs.projectcalico.org/manifests/custom-resources.yaml

Wait for the calico-node pods to be created. Note that some Calico components (e.g. calico-kube-controllers) will not be able to start until you’ve completed the next step—that’s normal.

Remove leftover Cilium CNI config files from nodes

The Cilium install will have left some files on the nodes. This includes a CNI configuration file, which we need to remove so that kubelet will call the Calico configuration instead:

$ kubectl get po -n calico-system | grep calico-node | awk '{print $1}' | xargs -I {} kubectl exec -n calico-system {} -c calico-node -- rm -f ./host/etc/cni/net.d/05-cilium.conflist

All Calico pods should now transition to a Ready state when they next try to start.

Reboot nodes

Reboot only the Kubernetes node VMs (you do NOT need to restart the etcd VMs). You can do this using “restart Guest OS” in the vSphere UI. This step clears out any stale Cilium BPF programs and restarts all pods using Calico networking.

Possible next steps

Run Cyclonus

Cyclonus is a network policy conformance tool to check that your cluster is able to implement all of Kubernetes network policy. The quickstart document can be found here.

Deploy a microservices demo application

If you don’t already have a microservice application of your own to run on the cluster, you could try this demo application:

kubectl apply -f http://raw.githubusercontent.com/microservices-demo/microservices-demo/master/deploy/kubernetes/complete-demo.yaml

This demo opens a Nodeport service on port 30001, so point your browser at the IP of one of your nodes on port 30001. For example: http://10.12.2.178:30001/

Run a benchmark

This benchmark can be used to test the network performance of your cluster.

Enable BPF mode

Calico has multiple data planes available. To try out Calico’s eBPF data plane, follow these instructions.

Try Calico Cloud

Calico Cloud is a pay-as-you-go security and observability SaaS platform for containers, Kubernetes, and cloud. Get a live view of dependencies and how all the services are communicating with each other in a multi-cluster, hybrid, and multi-cloud environment. Eliminate setup and onboarding steps and troubleshoot your Kubernetes security and observability issues within minutes.