Ephemeral Clusters as a Service with ClusterAPI and GitOps

The presentation discusses the challenges of scaling observability and deployment automation in GitOps and proposes a solution using open-source tools like ClusterAPI, ArgoCD, and Prometheus+Thanos to manage and organize deployments.

• GitOps has clear advantages over traditional CI/CD tools, but scaling observability and deployment automation can be challenging
• Open-source tools like ClusterAPI, ArgoCD, and Prometheus+Thanos can help manage and organize deployments
• The presentation proposes using immutable clusters treated as always stamped out of a template to address the fear of upgrading
• The ClusterAPI project can be used to declaratively express the idea of a cluster and interact with different cloud providers
• The v-cluster project can be used to create ephemeral clusters that live inside management clusters and can be used when provisioning time is a crucial parameter
• The presentation emphasizes the importance of monitoring ephemeral clusters and collecting metrics from them
• The use of open-source tools can automate the deployment of hundreds of clusters and applications automatically and securely

The presenters shared their experience working with large-scale customers who were afraid of upgrading clusters and deploying new versions on existing clusters. This fear drove them to propose the idea of immutable clusters treated as always stamped out of a template, which can be replaced with new clusters when needed. They also emphasized the importance of monitoring ephemeral clusters and collecting metrics from them, which can be achieved using open-source tools like ClusterAPI, ArgoCD, and Prometheus+Thanos.

GitOps has seen widespread adoption in the last few years due to the clear advantages over traditional CI/CD tools. However, with adoption comes the growing pains of scale: running and managing multiple clusters across different cloud providers represents a major hurdle for organizations wanting to adopt Kubernetes as a standard deployment platform. In particular, observability and security at scale are two thorny aspects that need to be addressed; we will demonstrate how it’s possible to tame the complexity of such scaled infrastructure via open-source tools, such as ClusterAPI, ArgoCD and Prometheus+Thanos to provide control and visibility over an arbitrary number of clusters. We will show a sample, created after our collective experience at large scale customers, which can automate the deployment of hundreds of clusters and applications automatically and securely, and collect metrics from all the ephemeral clusters along the way.