--- title: "Another Day, Another Lesson in Securing Your Infrastructure" source: "https://www.tigera.io/blog/another-lesson-in-securing-your-infrastructure/" --- [Technical Blog](https://www.tigera.io/category/technical-blog/) # Another Day, Another Lesson in Securing Your Infrastructure By [Christopher Liljenstolpe](https://www.tigera.io/blog/author/chris-liljenstolpe/) on Mar 22, 2018 • 3 min read Last week I wrote a [blog post](https://www.tigera.io/blog/tesla-compromise-network-policy/) that was a response to an unfortunate event at Tesla.  In it, I detailed some techniques one could use to secure the infrastructure to reduce the risk of suffering a similar security ‘event.’ One of the points I made in that post was that security isn’t “just for production” as production is increasingly a “fuzzy” concept between PoC, PoC with trial customer(s), first application deployment, and so forth. Right on the heels of that blog post, security researcher [Giovanni Collazo](http://elweb.co/author/gcollazo/) published [this blog post](http://elweb.co/the-security-footgun-in-etcd/) where he details a fairly simple search for publicly accessible [etcd](http://coreos.com/etcd/) servers on the Internet.  etcd does not ship with user authentication turned on by default.  If you can find an etcd server on the network with a default configuration, you can interrogate it and potentially modify its contents. Ok, I imagine many of you are wondering “Why this is important, and what is etcd?”  This is where this gets interesting – etcd is a database that was designed to hold configuration data for other servers and services.  The folks at [CoreOS](http://coreos.com/) (soon to be part of Red Hat) initially wrote etcd to help automate the (re)-configuration of servers and services to ensure that they are kept up-to-date.  This is an admirable, security-driven initiative. It was so admirable that many projects now include etcd as part of their infrastructure, not the least of which is [Kubernetes](http://kubernetes.io/), the very popular container orchestration system that came out of Google and is now the darling of the cloud-native world. Kubernetes stores lots of “really interesting” data in etcd, as do lots of other projects and users.  In the 2000+ public etcd servers that Giovanni found, he located almost 9000 keys labeled “password” and over 650 keys labeled “aws_secret_access_key.”  I’ll pause here for a minute to allow you to go run and check your etcd servers…. If you install etcd in a default manner, you are basically raining your newly compromised keys on an appreciative, if somewhat unscrupulous, audience. Let me reiterate the point I made earlier here, and in the previous blog.  **Security is not just for production!**  If you don’t PoC what you plan to deploy, it is most likely that you will deploy what you PoC’d. A best practice is to evaluate the components that make up your infrastructure and your application environment and make sure they have rational security settings.  Enabling user authentication is good, enabling mutual TLS (mTLS) authentication for RESTful and similar services (such as etcd) is even better. With [Project Calico](https://www.tigera.io/project-calico/)’s [network policy](https://www.tigera.io/blog/micro-segmentation-in-the-cloud-native-world-355d2591cb6d/), and specifically [host endpoints](https://www.tigera.io/blog/securing-host-endpoints-with-calico-part-3/), as I discussed in the [previous post](https://www.tigera.io/blog/tesla-compromise-network-policy/), you could isolate what endpoints could even connect to your etcd servers.  With [Tigera’s CNX](https://www.tigera.io/tigera-products/compare-products/), you could go further and control who within your organization can change the policies that control etcd access. Security starts with some basic fundamental hygiene, like understanding what it is that you are using, how it deploys by default, and what tools are inherent in the component that can improve your security posture.  From there you can build on those basic protections with defense in depth including network policy. Doing this at the beginning of a deployment is easier, and certainly less disruptive than trying to patch after the fact.  Unfortunately, patching after someone has pulled your “aws_secret_access_key” from your open etcd server is not only disruptive, but also ineffective. The re-keying necessary to recover from that sort of incident (assuming you detect it at all) is more disruptive still. [Best Practices](https://www.tigera.io/tags/best-practices/)[Open Source](https://www.tigera.io/tags/open-source/)[Project Calico](https://www.tigera.io/tags/project-calico/) ## Related posts [![From Merged PR to Governed Pod: Automating AI Agent Deployment on Kubernetes](https://www.tigera.io/app/uploads/2026/10/From-Merged-PR-to-Governed-Pod-featured.png)](https://www.tigera.io/blog/from-merged-pr-to-governed-pod-automating-ai-agent-deployment-on-kubernetes/) #### [From Merged PR to Governed Pod: Automating AI Agent Deployment on Kubernetes](https://www.tigera.io/blog/from-merged-pr-to-governed-pod-automating-ai-agent-deployment-on-kubernetes/) By [Alister Baroi](https://www.tigera.io/blog/author/alister-baroi/) on Oct 13, 2026 In short, automating the AI agent lifecycle is this year’s pitch: Google Cloud’s walkthrough of 30th July 2026 turns the whole thing into a conversation with a coding agent. On Kubernetes the deploy half of... [Read more](https://www.tigera.io/blog/from-merged-pr-to-governed-pod-automating-ai-agent-deployment-on-kubernetes/) [![Gemini Never Left the Sandbox. The Sandbox Had a Door.](https://www.tigera.io/app/uploads/2026/10/gemini-featured.png)](https://www.tigera.io/blog/gemini-never-left-the-sandbox-the-sandbox-had-a-door/) #### [Gemini Never Left the Sandbox. The Sandbox Had a Door.](https://www.tigera.io/blog/gemini-never-left-the-sandbox-the-sandbox-had-a-door/) By [Alister Baroi](https://www.tigera.io/blog/author/alister-baroi/) on Oct 7, 2026 In short, in May 2026 a Gemini model under evaluation by the AI security firm Irregular reached the systems of three real companies, and the incident has been reported as a breakout. By Google’s own... [Read more](https://www.tigera.io/blog/gemini-never-left-the-sandbox-the-sandbox-had-a-door/) [![EU Cyber Resilience Act: Europe Just Put Your AI Agents on a 24-Hour Clock](https://www.tigera.io/app/uploads/2026/10/EU-Cyber-Resilience-Act-AI-Agents-featured.png)](https://www.tigera.io/blog/eu-cyber-resilience-act-europe-just-put-your-ai-agents-on-a-24-hour-clock/) #### [EU Cyber Resilience Act: Europe Just Put Your AI Agents on a 24-Hour Clock](https://www.tigera.io/blog/eu-cyber-resilience-act-europe-just-put-your-ai-agents-on-a-24-hour-clock/) By [Alister Baroi](https://www.tigera.io/blog/author/alister-baroi/) on Oct 5, 2026 In short, the EU Cyber Resilience Act (CRA) puts binding cybersecurity requirements on any software sold as a product in the EU, and it does not carve out AI agents. Since 11th September 2026, any... [Read more](https://www.tigera.io/blog/eu-cyber-resilience-act-europe-just-put-your-ai-agents-on-a-24-hour-clock/)