Using Calico to create a Kubernetes cluster mesh for multi-cluster environments

Kubernetes has come of age with more organizations adopting a microservices architecture at scale. But scale brings a whole slew of new challenges, especially with Kubernetes, which is designed to operate as a single cluster. However, the usage of Kubernetes, especially at leading-edge organizations operating at scale, has crossed the single-cluster threshold. Organizations are building and deploying services across multiple clusters for high availability, disaster recovery, application isolation, compliance, latency concerns, staged migration, and multi-tenancy reasons.

Regardless of the reasons to deploy multiple clusters, platform and application teams must address networking, security, and observability issues related to microservices deployed across multi-clusters, sometimes spanning hybrid and multi-cloud environments.

Calico, the most widely adopted container networking and security solution (according to a recently published container adoption report by Datadog), provides an operationally simple solution to solve the networking, security, and observability challenges of running multi-cluster Kubernetes environments.

Security, observability, and networking requirements for multiple Kubernetes clusters

In simple terms, creating a multi-cluster Kubernetes environment requires stitching multiple Kubernetes clusters together to provide a common set of services. To create a single logical environment spanning multiple clusters, the key requirements are:

  • Enabling inter-cluster communication – Communication across pods located in different clusters is essential for all use cases for microservices spanning multiple clusters.
  • Service discovery – To enable the consumption of services deployed in different clusters, each cluster must be aware of the workloads deployed in the other clusters.
  • Load balancing – If a multi-cluster architecture is used to deploy highly scalable and available services, it should be possible to load balance the traffic across multiple clusters.
  • Network and security policy – Network and security policies that protect access to and from workloads, as well as help achieve microsegmentation, should extend to workloads deployed across multiple clusters.
  • Compliance – Compliance is another challenge when working with a multi-cluster environment. For example, GDPR may limit communication between clusters based on the geographic location of the clusters. In that case, the security policies governing compliance in a multi-cluster environment must ensure the right level of isolation.
  • Encryption – Encryption of communication within and across clusters is often required as part of internal security controls as well as some compliance regulations.
  • Observability – Observability is challenging in multi-cluster architectures where services are deployed across multiple clusters in hybrid and multi-cloud environments.

How Calico enables a Kubernetes cluster mesh for security, observability, and networking in multi-cluster environments

Calico provides a centralized management plane and single point of control for security, observability, and networking for a multi-cluster Kubernetes deployment. Let’s take a closer look at Calico’s capabilities in this regard.

Centralized management plane for multiple clusters

Managing a standalone cluster with multiple nodes is not onerous; however, when you add multiple clusters, the complexity compounds. Calico provides a multi-cluster management plane to enable security and observability across multiple clusters in hybrid and multi-cloud environments, and user access control using RBAC. This architecture also supports the federation of network policy resources across clusters and lays the foundation for a single pane of glass.

Federation

With Calico, you can create policies in one cluster that reference pods in another cluster using federated identity. Federated services provide service discovery of remote pods in another cluster. With these two features, you can define fine-grained security controls between multiple clusters.Two hexagons are next to each other on a white background, labeled as Cluster A and Cluster B respectively. Four dots are in each hexagon. A dot in Cluster B is connected to a dot in Cluster B with a straight line.

Service discovery using federated services

Calico supports the creation of a federated service that consolidates services from different endpoints. A federated service (also called a backing service) is a set of services with consolidated endpoints. Calico discovers services across all clusters (both local and remote clusters) and creates a federated service on the local cluster that encompasses all individual services.

Federated tiers and policies

Using federated tiers and federated policies, you can define security policies that apply across all clusters, or to a specific group of clusters. If you plan to deploy multiple clusters, federated tiers, and policies, you can extend your security controls to each existing and new cluster. This reduces duplication of policies (and maintenance of identical policies per cluster) to simplify the creation and maintenance of your security controls.

Unified security policy controls for application and network layers that work for local and federated microservices 

Calico applies policies on federated services in the same way it applies policies on local services within a cluster, thus creating a seamless set of policies that authorize access to both local and remote microservices. Policy definitions within a cluster can reference labels on workload and host endpoints from remote clusters. Calico policies can be applied to both local and remote services. Calico’s policy recommender will factor communications with remote cluster workloads and can generate security policy rules to permit inter-cluster communications.

Unified security policy controls across bare metal, hosts, VMs, and containers

In the complex world of digital transformation, application modernization often happens in phases over a long period of time. During this time, it is common to have legacy monolithic services on VMs and hosts, run along more modern microservices in containers. Calico provides a unified policy framework that works across bare metal, hosts, VMs, and containers enabling the co-existence of legacy and modern environments.

Encryption

Calico offers encryption for data in transit that leverages the latest in crypto technology, using open-source WireGuard. As a result, Calico’s encryption is highly performant and leverages an architecture that still allows visibility into workload communication.

Observability and troubleshooting

Calico offers visibility into service-to-service communication in a way that is resource efficient and cost effective. It provides Kubernetes-native visualizations of all the data it collects, in the form of a Dynamic Service and Threat Graph. The graph allows the user to visualize communication flows across services and team spaces, to facilitate troubleshooting. This is beneficial to platform operators, service owners, and development teams.

With Calico, Envoy is integrated into the data plane to provide operational simplicity. No matter which data plane you’re using (standard Linux iptables, Windows, or eBPF), Calico provides observability, traffic flow management, and control by deploying a single instance of Envoy as a daemon set on each node of your cluster. Instead of having an Envoy container in every pod as a sidecar, there is one Envoy container per node. This provides performance advantages, as it’s more resource-efficient and cost-effective.

For federated services, the Dynamic Service and Threat Graph can visualize microservices communications across clusters. This visibility can easily be extended to workloads outside the cluster in other remote clusters as well.

In addition to the Dynamic Service and Threat Graph, the flow logs generated by Calico are enriched with contextual information on both local and remote cluster workloads for inter-cluster communication.

Conclusion

Calico offers an elegant and operationally simple solution for organizations running Kubernetes at scale. The ability to create a Kubernetes cluster mesh that enables networking, security, and observability unlocks the scale at which microservices can be operated. Some of the advantages Calico offers for building a Kubernetes cluster mesh include:

  • No need for an additional management plane
  • Sidecar-less data plane with Envoy as a DaemonSet
  • Multi-cluster federation across hybrid and multi-cloud
  • Multi-cluster observability from network to application layer
  • Blazing-fast encryption
  • Industry’s most advanced zero-trust security policy engine

Download Calico for Kubernetes cluster mesh whitepaper

Getting started

Get hands-on experience with using Calico for Kubernetes multi-cluster mesh. Try now.

Join our mailing list

Get updates on blog posts, workshops, certification programs, new releases, and more!