Sam Aydlette

Listen up, cloud enthusiasts and container aficionados! Today I'm taking the gloves off and we're diving into the controversial topic of inventory management. I know what you're thinking: "Inventory management? That's so legacy IT!" Well buckle up, because I'm about to change your mind faster than a container can restart itself.

The Hard Truth: It's Container Chaos Out There

Let's be honest: many Kubernetes environments look like my living room after I put my kids to bed each night - things everywhere, no idea where they came from, and a vague sense that something important might be shoved under the couch cushions. You're deploying containers left and right, patting yourself on the back for being "cloud-native," all while having absolutely no idea what's actually running in your environment.

Sound familiar? If so, you're definitely not alone.

Here's the thing: maintaining an accurate inventory isn't some outdated practice from the dark ages of physical servers. It's the cornerstone of effective system governance, establishing clear system boundaries and data flows. It's the map that makes everything else possible - security, compliance, operations, the whole shebang.

"But Containers Are EPHEMERAL!"

I can hear some of you typing furiously in the comments already:

"Containers are EPHEMERAL by design! They come and go! That's the whole point! Why inventory something that's designed to disappear? Next you'll tell me I need to document every AWS Lambda execution!"

Let me pour you a steaming hot cup of reality.

Yes, containers are ephemeral. But just because something doesn't stick around forever doesn't mean you don't need to know it exists. It only takes an attacker a moment to exploit an open vulnerability. And general purpose, non-hardened container images are riddled with vulnerabilities.

In fact, the ephemerality of containers is even more reason to maintain robust inventory practices. Because they come and go so quickly, you need automated ways to track:

• What images are running
• Where they came from
• Who deployed them
• How they are configured
• What secrets they're accessing
• What data they're processing

Without this visibility you're essentially running a "security through obscurity" strategy, except the person left in the dark is yourself. As I described in Part 1, attackers can simply use tools like Nmap and Shodan to know exactly what you're running (as well as vulnerabilities and misconfigurations that may be present).

How to Track a Container Inventory

Fortunately, there are several viable approaches to maintaining an accurate container inventory:

1. Cloud Platform Services: AWS, GCP, and Azure all offer specialized container inventory management tools.

AWS Services:

• Amazon Elastic Container Registry (ECR): A managed Docker container registry.
• Amazon Elastic Container Service (ECS): A managed container orchestration service.
• Amazon Elastic Kubernetes Service (EKS): A managed Kubernetes service. EKS offers more granular control over container deployment through Kubernetes pods.
• AWS Config: Provides a detailed view of AWS resources and their configurations, including container-related resources. This service supports ECS, ECR, and EKS, allowing you to track configuration changes to container assets and maintain compliance.
• AWS Systems Manager: Helps manage container instances with inventory capabilities that track installed applications, configuration details, and patch compliance status.
• AWS Inspector: Automatically discovers container images in Amazon ECR repositories and immediately begins scanning them for vulnerabilities with no manual configuration required.

Google Cloud Platform (GCP) Services:

• Google Cloud Asset Inventory: Provides inventory services based on a time series database that maintains a history of Google Cloud asset metadata. This service allows you to track containers and related resources across your GCP environment.
• Google Kubernetes Engine (GKE): A managed Kubernetes service that automatically integrates with Cloud Asset Inventory to maintain visibility into your container deployments.
• Google Container Registry (GCR) and Artifact Registry: Secure container image repositories that work with vulnerability scanning tools to maintain an inventory of images and their security status.
• Google Cloud Monitoring: Provides insights into the health, performance, and availability of your containerized applications.

Microsoft Azure Services:

• Azure Container Registry (ACR): A managed registry service where you can store and manage container images and related artifacts.
• Azure Kubernetes Service (AKS): A managed Kubernetes service that integrates with Azure Monitor and other Azure services for comprehensive inventory management.
• Microsoft Defender for Containers: Provides comprehensive inventory capabilities that enable you to explore containerized assets across multicloud and on-premises environments.
• Azure Resource Graph: Allows you to query container resources and their configurations across your Azure subscriptions, providing detailed inventory insights.

2. Kubernetes Custom Resource Definitions (CRDs): For those running on-premises or hybrid environments, Kubernetes CRDs provide a powerful approach to inventory management. They allow you to extend the Kubernetes API to track physical host information alongside container deployments.

I'm particularly fond of the CRD approach because it lets you stay within the Kubernetes ecosystem. You can use familiar kubectl commands to query your entire container inventory, visualize deployment patterns, and enforce compliance requirements.

Here's how you might implement inventory management with CRDs:

Define your custom resource: You create a CRD called ContainerImage that tracks key information like:

• Image name and version
• Registry source
• Deployment locations (which pods/namespaces)
• Creation timestamps
• Security scan results

Create a controller: Deploy a small Python controller that watches for pod creation/deletion events across your cluster. Whenever a container is deployed or removed, it automatically updates your inventory records.

Query your inventory: With this system in place, your security team can run commands like:

Integrate with deployment and security tools: Container inventory becomes even more powerful when integrated into a GitOps workflow. By storing your inventory definitions and policies as code in Git, you create a single source of truth for both what should exist and what actually exists in your environment.

This closed-loop system where inventory definitions live in Git, deployments are automated through pipelines, and inventory controllers continuously reconcile desired state with actual state provides the ultimate defense against configuration drift and unauthorized deployments.

When a security scanner flags a vulnerable container, your GitOps pipeline can automatically create a pull request to update the affected deployments, ensuring that intended changes flow through the same auditable process as all other infrastructure modifications.

The same is true for SecOps. For example, your SOC team connects this inventory to your SIEM. Now when an alert fires, they immediately know what image was involved, its deployment history, and whether it contains known vulnerabilities.

What makes this approach powerful is that it works uniformly across your entire environment. The same kubectl commands work whether the container is running in your AWS EKS clusters or your on-premises data center. Your teams use familiar Kubernetes tools and don't need to learn a bunch of different inventory management systems.

Conclusion: You Are What You are Running

In the fast-paced world of cloud computing, knowing what you are running is the first step to managing it effectively. Yes, containers are ephemeral, but that's all the more reason to keep track of them. Think of your container inventory as less of a static document and more of a cybernetic feedback loop - just like your container environment itself.