--- title: "kr8 – Configuration Management for Kubernetes Clusters" source: "https://www.tigera.io/blog/kr8-configuration-management-for-kubernetes-cluster/" --- [Technical Blog](https://www.tigera.io/category/technical-blog/) # kr8 – Configuration Management for Kubernetes Clusters By [Lee Briggs](https://www.tigera.io/blog/author/lee-briggs/) on Jun 27, 2019 • 5 min read This article originated from [http://leebriggs.co.uk///blog/2018/11/07/kr8-kubernetes-config-mgmt.html](http://leebriggs.co.uk///blog/2018/11/07/kr8-kubernetes-config-mgmt.html) Previous visitors to this blog will remember [I wrote about configuration management for Kubernetes clusters](http://leebriggs.co.uk/blog/2018/05/08/kubernetes-config-mgmt.html), and how the space was lacking. For those not familiar, the problem statement is this: it’s really hard to maintain and manage configuration for components of multiple Kubernetes clusters. As the number of clusters you have starts to scale, keeping the things you need to run in them (such as ingress controllers) configured and in sync, as well as managed the subtle differences that need to be managed across accounts and regions. With that in mind, it’s my pleasure to announce that at my employer, [Apptio](http://www.apptio.com) we have tried to solve this problem with [kr8](http://github.com/apptio/kr8). kr8 is an opinionated Kubernetes cluster configuration management tool, designed to be simple, flexible and use off the shelf tools where possible. This blog post details some of the design goals of kr8, as well as some of the benefits and a few examples. ### Design Goals The intention when making kr8 was to allow us to generate manifests for a variety of Kubernetes clusters, and give us the ability to template and override yaml parameters where possible. We took inspiration from a variety of different tools such as [Kustomize](http://github.com/kubernetes-sigs/kustomize), [Kasane](http://github.com/google/kasane), [Ksonnet](http://github.com/ksonnet/ksonnet) and many others on our journey to creating a configuration management framework that is relatively simple to use, and follows some of the practices we’re used to as Puppet administrators. Other design goals included: - No templating engine - Flexibility - Compatibility with existing deployment tools, like [Helm](http://www.helm.sh/) - Small binaries, with off the shelf tools used where needed The end goal was to be able to take existing helm charts, or yaml installation manifests, then manipulate them to our needs. We chose to use [jsonnet](http://jsonnet.org/) as the language of kr8 due to its data templating capabilities and ability to work with both JSON and YAML. ### Terminology & Tools #### kr8 kr8 itself is the only component of the kr8 framework that we wrote at Apptio. Its purposes are: - Discover clusters in a [hierarchical directory tree](http://github.com/apptio/kr8-configs/tree/master/clusters) - Discover components in a [components directory](http://github.com/apptio/kr8-configs/tree/master/components) - Map components to clusters, using a [cluster.jsonnet](http://github.com/apptio/kr8-configs/blob/master/clusters/gke/cluster.jsonnet) file You can see the result of these purposes using a few of the tools in the kr8 binary, for example, listing clusters: ![kr8 cluster list showing cluster names (do, gcloud, etc.) and their config file paths, illustrating Kubernetes cluster](/app/uploads/2019/06/Screenshot-45.png) However, using the kr8 binary alone is probably not what you want to do. We bundle and use a variety of other tools with kr8 to achieve the ability to generate manifests for multiple clusters and deploy them. ### Task [Task](http://github.com/go-task/task) does a lot of the heavy lifting for kr8. It is a task runner, much like [Make](http://www.gnu.org/software/make/) but with a more flexible DSL (yep, it’s yaml/json!) and the ability to run tasks in parallel. We use Taskfiles for each component to allow us to build the component config. This gives us the flexibility to use rendering options for each component that make sense, whether it be pulling in a Helm chart or plain yaml. We can then input that yaml with kr8, and manipulate it with jsonnet code to add, modify the resulting kubernetes manifest. Alongside this, we use a taskfile to generate deployment tasks and to generate *all* components for a Task. This gives us the ability to execute lots of generate manifest jobs in relatively short periods of time. ### Kubecfg We use [Kubecfg](http://github.com/ksonnet/kubecfg) to do the actual deployment of these manifests. Kubecfg gives us the ability to validate, diff and iteratively deploy Kubernetes manifests which `kubectl` does not. The kubecfg jobs are generally inside Taskfiles at the cluster level. ### Components Components are very similar to helm charts. They are installable resource collections for something you’d like to deploy to your Kubernetes clusters. A component has 3 minimal requirements: - `params.jsonnet`: contains configurable params for the component - `Taskfile.yml`: Instructions to render the component - An installation source: This can be anything from a pure jsonnet file to a helm input values file. Ultimately, this needs to be able to generate some yaml I intend to write many more kr8 blog posts and docs, detailing how kr8 can work, examples of different components and tutorials. Until then, take a look at these resources: - [kr8](http://github.com/apptio/kr8) binary - [kr8-configs](http://github.com/apptio/kr8-configs) example repo - [cluster_config](http://github.com/jaxxstorm/cluster_config) my cluster configuration repo ### Acknowledgements I want to thank Colin Spargo, who came up with the original concept of kr8 and how it might work, as well as contributing large amounts of the kr8 code. I also want to thank Sanyu Melwani, who had valuable input into the concept, as well as writing many kr8 components. Finally, a thank you to our employer, Apptio, who has allowed us to spend time creating this tool to ease our Kubernetes deployment frustrations. If you’re interested in working on fun projects like this, we are [hiring for remote team members](http://www.apptio.com/about/careers/job-openings) [Lee Briggs](http://www.linkedin.com/in/briggsl/) is a Tigera guest blogger. He is an experienced DevOps Engineer Specialties: Redhat, Unix, Linux, Monitoring, Kubernetes, AWS,Containers, Hashicorp, DevOps, SRE ————————————————- [**Free Online Training**](https://www.tigera.io/events/) Access Live and On-Demand Kubernetes Tutorials [**Calico Enterprise – Free Trial**](https://www.calicocloud.io/home) Solve Common Kubernetes Roadblocks and Advance Your Enterprise Adoption [Open Source](https://www.tigera.io/tags/open-source/) ## Related posts [![What’s new in Calico: Spring 2026 Release](https://www.tigera.io/app/uploads/2026/06/Whats-New-in-Calico-NEW-TEMPLATE-2026.png)](https://www.tigera.io/blog/whats-new-in-calico-spring-2026-release/) [Company Blog](https://www.tigera.io/category/company-blog/) #### [What’s new in Calico: Spring 2026 Release](https://www.tigera.io/blog/whats-new-in-calico-spring-2026-release/) By [Veronika Smolik](https://www.tigera.io/blog/author/veronika-smolik/) on Jun 2, 2026 Kubernetes has come a long way since its debut in 2014. It’s gone from running a couple of containerized microservices to orchestrating fleets of production workloads spanning everything from AI agents to full scale VMs... [Read more](https://www.tigera.io/blog/whats-new-in-calico-spring-2026-release/) [![Kubernetes Operational Maturity: Secure and Resilient Cluster Federation with Cluster Mesh](https://www.tigera.io/app/uploads/2026/05/Kubernetes-Operational-Maturity-Secure-and-Resilient-Cluster-Federation-with-Cluster-Mesh.png)](https://www.tigera.io/blog/kubernetes-operational-maturity-secure-and-resilient-cluster-federation-with-cluster-mesh/) #### [Kubernetes Operational Maturity: Secure and Resilient Cluster Federation with Cluster Mesh](https://www.tigera.io/blog/kubernetes-operational-maturity-secure-and-resilient-cluster-federation-with-cluster-mesh/) By [Veronika Smolik](https://www.tigera.io/blog/author/veronika-smolik/) on May 25, 2026 Practically no one runs a single Kubernetes cluster in production these days. Maybe that’s how it started but data sovereignty requirements, acquisitions, AI initiatives and the need for edge servers, among other considerations, have pulled... [Read more](https://www.tigera.io/blog/kubernetes-operational-maturity-secure-and-resilient-cluster-federation-with-cluster-mesh/) [![What’s New in Calico v3.32](https://www.tigera.io/app/uploads/2026/05/Green-Please-use-a-different-background-color-alternately-1.png)](https://www.tigera.io/blog/whats-new-in-calico-v3-32/) #### [What’s New in Calico v3.32](https://www.tigera.io/blog/whats-new-in-calico-v3-32/) By [Reza Ramezanpour](https://www.tigera.io/blog/author/rezar/) on May 13, 2026 We’re excited to announce the release of Calico Open Source v3.32! 🎉 This release corresponds with Kubernetes v1.36 (Codename Haru) and it goes beyond just sharing a cat as the mascot of the release, it... [Read more](https://www.tigera.io/blog/whats-new-in-calico-v3-32/)