Early adopter enterprises across verticals such as Retail, Manufacturing, Oil & Gas are looking to incorporate containers and Kubernetes as a way of modernizing their applications. Choosing k8s as a standard ensures that these applications can be deployed these on different data center infrastructures (bare metal/VMware/KVM on OpenStack etc) or on public clouds (AWS/Azure/GCP etc). At times due to business & compliance reasons, some of these applications may need to be isolated and not connected to the outside world – a situation termed as ‘air-gapping’.  There are complex technical challenges associated with the deployment and operations of K8s clusters in air-gapped environments.
The last blog discussed – https://www.tigera.io/blog/containerized-air-gapped-edge-platform-architecture/ – air gapped deployments and the applicability of kubernetes as the foundational layer for such architectures.
I am excited to announce that thanks to work by two of my colleagues Arun Sriraman and Dan Lipovetsky,  Klusterkit, a set of three open source tools, is now available under the Apache v2.0 license on GitHub.[1]
The below section has been authored by Arun and Dan[2].

Introducing KlusterKit

Klusterkit is composed of three independent tools, that can be used separately, or in tandem to orchestrate the lifecycle of a production-grade Kubernetes cluster.
These include:
  1. etcdadm, a CLI that simplifies operating an etcd cluster
  2. nodeadm, a CLI node administration tool that complements kubeadm by deploying the dependencies that kubeadm requires
  3. cctl, a cluster lifecycle management tool that adopts the Kubernetes community’s Cluster API and uses nodeadm and etcdadm to easily deploy and maintain highly-available Kubernetes clusters in on-premises, even air-gapped environments.

 

The combination of these three tools enables:
  • Easy deployment and operating of highly-available etcd cluster and Kubernetes control plane in air-gapped, on-premise environments using the Cluster API.
  • Easy recovery from failures of the cluster control plane using etcd backup
  • Packaging of all the artifacts required to deploy Kubernetes in an air-gapped environment.

 

cctl

We have already covered etcdadm in a previous blogpost as shown below.
cctl is a cluster lifecycle management tool that adopts the Kubernetes community’s Cluster API and uses nodeadm and etcdadm to easily deploy and maintain highly-available Kubernetes clusters in on-premises, even air-gapped environments.[2]
Along with etcdadm and nodeadm, this tool makes up klusterkit, which lets you create, scale, backup and restore your air-gapped, on-premise Kubernetes cluster.

cctl Features

  • Highly-available Kubernetes control plane and etcd
  • Deploy & manage secure etcd clusters
  • Works in air-gapped environments
  • Rolling upgrade support with rollback capability
  • Flannel (vxlan) CNI backend with plans to support other CNI backends
  • Backup & recovery of etcd clusters from quorum loss
  • Control plane protection from low memory/cpu situations

 

Klusterkit Features

  • Multi-master (K8s HA) support
  • Deploy and manage secure etcd clusters
  • Works in air-gapped environments
  • Rolling upgrade and rollback capability
  • Flannel (vxlan) CNI backend with plans to support other CNI backends
  • Backup and recovery of etcd clusters from quorum loss
  • Control plane protection from low memory/low cpu conditions

 

 Architecture

The architecture of Klusterkit is as shown above. All of the metadata of the cluster is stored in a single yam file -cctl-state.yaml. A key reqiuirement of air-gapped deployments is that operators are not necessarily technically skilled. Thus, the cctl CLI can be invoked to orchestrate the lifecycle of a Kubernetes cluster from any machine which contains this state file. The machine could be an operator’s laptop or any of the machines that are part of the Kubernetes cluster.
Cctl implements and calls into the upstream community cluster-api interface as a library for CRUD operations on clusters. As shown above, It uses “ssh-provider”, the open-source bare-metal cluster-api provider authored by Platform9, which in-turn calls etcdadm and nodeadm to perform cluster operations.

How to use Klusterkit and its components:

  1. Each of the three tools can be easily built by calling “go get” as shown below:
  2. These binaries can then be packaged and copied over to the target machines that need to run a highly available Kubernetes cluster. Place the nodeadm and etcdadm binaries in the versioned directories:
    cp $GOPATH/bin/nodeadm /var/cache/ssh-provider/nodeadm//
    cp $GOPATH/bin/etcdadm /var/cache/ssh-provider/etcdadm//
  3. If the intention is to orchestrate a Kubernetes cluster on-prem, in an air-gapped environment, nodeadm and etcdadm download command makes is easy to pull down required dependencies beforehand from a machine with internet connectivity. These downloaded bits (i.e. kubelet and kubelet systemd file, kubeadm binary, CNI binaries, all container images including Kubernetes, keepalived image and systemd file, etcd container image and related configuration files), can then be copied over to the air-gapped hosts along with cctl, nodeadm, and etcdadm. (For more information see wiki).
  4. Once all these pieces are in-place, bringing up your first Kubernetes cluster is a couple of commands away:
    • First, create the credentials used for the cluster.
      $GOPATH/bin/cctl create credential –user root –private-key ~/.ssh/id_rsa
    • Then, create a cluster object. Use –help to see a list of supported flags.
      $GOPATH/bin/cctl create cluster –pod-network 192.168.0.0/16 –service-network 192.169.0.0/24
    • Finally, create the first machine in your cluster.
      $GOPATH/bin/cctl create machine –ip $MACHINE_IP –role master

References:

  1. Introducing KlusterKit –https://platform9.com/blog/introducing-klusterkit-an-open-source-toolkit-to-simplify-kubernetes-deployments-on-premise-air-gapped-environments/
  2. Github – https://github.com/platform9/cctl/
  3. See the technical introduction to nodeadm
  4. Learn more about  etcdadm 
  5. Join the CNCF-hosted webinar hosted on, 4/16/2019 for a deep-dive into etcdadm – one of the tools included in Klusterkit. Register here.

This article originated from http://www.vamsitalkstech.com/?p=7814

Vamsi Chemitiganti is a Tigera guest blogger. Vamsi Chemitiganti is Chief Strategist at Platform9 Systems. Vamsi works with Platform9’s Client CXOs and Architects to help them on key business transformation initiatives. He holds a BS in Computer Science and Engineering as well as an MBA from the University of Maryland, College Park.

Subscribe to our newsletter

Get updates on webinars, blog posts, new releases and more!

Thanks for signing up. You must confirm your email address before we can send you. Please check your email and follow the instructions.

Pin It on Pinterest

Share This