Datadog's unique approach to image signing and verification at scale in a Kubernetes environment
- Image signing and verification is crucial for securing the software supply chain and ensuring the integrity of container images
- Datadog's engineering teams use a wide variety of languages and CI/CD configurations, constantly deploying images to tens of thousands of nodes across dozens of Kubernetes clusters, spanning multiple cloud providers and datacenters
- To ease adoption and maintenance of image signing across heterogenous build environments, Datadog takes a service-oriented approach, encapsulating cryptographic complexity within a gRPC signing service
- To verify image signatures at runtime, Datadog uses an image verification plugin system contributed upstream to containerd, instead of using Kubernetes admission controllers
- Datadog's approach balances the need for fast developer feedback and better security properties
- Datadog's approach improves performance and reliability by diverting most of the registry load to the read path and avoiding introducing new cluster-level dependencies