Kubernetes is growing in popularity, and its use within organizations is maturing. According to the latest Cloud Native Computing Foundation (CNCF) annual survey, 83% of organizations surveyed use Kubernetes in production, and 28% of organizations have more than 11 Kubernetes production clusters.
At the same time, security concerns around Kubernetes are mounting. A Red Hat survey of Kubernetes adoption and security showed that, of 500 DevOps professionals surveyed:
Read on to better understand the security risks facing Kubernetes environments, understand what security mechanisms Kubernetes natively provides, and discover best practices for improving your Kubernetes security posture.
The following are some of the main risks facing Kubernetes production deployments.
To ensure the security of images, organizations should implement strong governance policies that ensure images are securely built and stored in trusted registries. For example, organizations should make sure that container images are built using pre-approved and secure base images, which should be regularly scanned for issues and vulnerabilities.
Organizations should standardize registries by creating a list of image registries that are allowed for use. Images should always be scanned before being used to create containers in a Kubernetes cluster, to avoid tampering.
Containers and pods need to communicate with each other as part of normal operations. However, this communication can be exploited by threat actors. A breached container can affect other containers and pods.
To ensure communication is secured, organizations should put in place network policies that limit communication to the minimum necessary for workloads to function. This includes both north-south traffic (ingress/egress traffic) and east-west traffic within the cluster. Network policies should be automatically adjusted to ensure they do not hurt productivity.
Visibility is critical to ensure security is maintained. However, it can be challenging to achieve visibility in complex, distributed, containerized environments.
Securing applications without visibility can be difficult or even impossible. Without visibility, it is not possible to detect critical vulnerabilities or discover misconfigurations before threat actors exploit them. Visibility is also critical to monitor containers at runtime to detect attacks, and to keep track of containers that are no longer in use in order to properly retire them before they become a liability.
Kubernetes is built to speed up the deployment of applications and simplify operations and management. While Kubernetes provides a wide range of controls that can help organizations effectively secure clusters and applications, it does not provide secure configurations out of the box.
For example, Kubernetes network policies behave similarly to firewall rules, controlling how pods can communicate with each other as well as other endpoints. Once a pod is assigned a network policy, it is only allowed to communicate with the assets declared in the network policy. However, Kubernetes does not apply network policies to pods by default. This means that after being deployed, all pods can talk to all other pods within a Kubernetes environment. This makes it critical to ensure all cluster resources have appropriate security policies defined.
Another concern is secret management. Secrets define how sensitive information, like keys and credentials, is accessed and stored. When managing secrets, it is critical to make sure that they are not used as environment variables, or hardcoded within images. Secrets should be managed externally to containers, with careful access control to protect secrets from unauthorized parties.
For many organizations, compliance is a critical concern. Achieving compliance in cloud-native environments is a highly challenging endeavor. To achieve compliance, organizations are usually required to implement certain security measures. This often requires enforcing best practices, benchmarks, and industry standards, as well as internal organizational policies.
In addition to maintaining compliance on a regular basis, organizations are also required to provide proof of compliance. This requires visibility, monitoring, and logs—the data needed to perform audits. To ensure auditors have access to the data or the systems, organizations need robust features that are not necessarily provided by plain, open-source Kubernetes.
Learn more in our guides to Kubernetes vulnerabilities and Kubernetes security issues
Cloud-native security is implemented across four key layers, known as the 4 C’s:
Kubernetes provides several concepts and mechanisms that can help secure your clusters. Here are the most important ones, as listed in the official Kubernetes project documentation.
Kubernetes uses a flat network model that allows each pod to communicate with any other pod in the cluster by default. This creates major security concerns, because it allows attackers who compromise one pod to freely communicate with all other resources in the cluster. To secure pod-to-pod communications, Kubernetes uses the concept of network policies.
A network policy is like a virtual firewall set up within each pod. When network policies are defined by an administrator, they are automatically updated at the pod level, and the pod accepts or rejects network traffic according to the policy. In Kubernetes, the mechanism for defining allowed or disallowed communication is label selectors, not IP addresses or ranges.
Kubernetes originally used the PodSecurityPolicy (PSP) object to control security-related pod configuration. A PSP defines the minimal conditions a pod must meet in order to run in the cluster. However, PSP has been found to be difficult to use, has important functional limitations, and cannot be enabled by default, and so will soon be deprecated and replaced.
The replacement feature has the temporary name “PSP Replacement Policy”. The new policy will be designed for simplicity of deployment, will have a built-in admission controller, and will be easy to use while providing flexibility for large-scale production deployments. Importantly, unlike the current PSP, you will be able to retrofit it to existing clusters via soft rollout.
Secrets are sensitive information like authentication tokens, SSH keys, or passwords. It is dangerous to store secrets in plaintext as part of container images or pod configurations, as these can easily be compromised by attackers who then gain access to any system accessible via those credentials.
The built-in Kubernetes Secrets feature lets you store sensitive information within a secret object in Kubernetes. You must first create a secret, populate it with secret data, update a service account to reference the secret, and only then create pods that use the secret. A pod can access a secret as an environment variable or a file (this is known as a secret volume).
Kubernetes uses the concept of ClusterRoles and Roles, which specify what each user can perform in a cluster or an entire Kubernetes namespace. You can use ClusterRoleBinding to apply a role to all resources in the cluster, or RoleBinding to apply the role to every resource in a namespace. Kubernetes offers the following default user-facing roles, and also lets you create additional custom roles:
The primary access point for a Kubernetes cluster is the Kubernetes API. Users and service roles access it via the kubectl utility, direct REST API requests, or client SDKs. By default, the API is accessed via port 445 and is protected by Transport Layer Security (TLS).
The Kubernetes API authentication process works as follows:
username, and can use this username in subsequent activities.Kubernetes Ingress is an object that allows services in a Kubernetes cluster to be accessed from outside the cluster. It facilitates connections via HTTP/S, and specifies routing rules.
To secure an Ingress object, you specify a secret containing two keys—tls.crt and tls.key—which contain the certificate and private key, respectively. TLS encryption works up to the ingress point, and traffic within the cluster is not encrypted by default. If the ingress configuration includes multiple hosts, they are multiplexed on the same port.
Kubernetes automatically assigns a Quality of Service (QoS) class to pods, which helps optimize scheduling and eviction of pods. The kubelet uses a QoS class to decide the order in which pods are evicted from nodes, and to make smarter decisions when scheduling pods on nodes. DevOps teams can customize QoS classes for pods if necessary.
You can place pods in a Guaranteed QoS class, which means that each container is guaranteed to receive enough resources from the node. This requires that all containers have a CPU and memory limit. Kubernetes assigns exactly enough CPU and memory resources to meet the requirements of containers in the pod.
Here are a few best practices that can help you secure Kubernetes clusters and workloads throughout the software development lifecycle.
Learn more in our blog post: What Your Kubernetes Security Checklist Might Be Missing
Related content: Kubernetes security policy design best practices
Tigera’s commercial solutions provide Kubernetes security and observability for multi-cluster, multi-cloud, and hybrid-cloud deployments. Both Calico Enterprise and Calico Cloud provide the following features for security and observability:
Security
Observability
Get updates on blog posts, workshops, certification programs, new releases, and more!
