Unified Chaos Injection And SLO Validation Experiments In Kubernetes

You have a principled process for releasing your Kubernetes app that involves load testing, benchmarking and validation of service-level objectives (SLOs). But, will your app perform well when your cluster is subject to compute, memory, i/o, or network stress? In this talk, we will explore a novel approach that combines chaos injection for probing weaknesses in your Kubernetes infrastructure, with load testing, benchmarking and performance validation with SLOs for your app. The core thrust of our approach will be flexibility combined with simplicity. Your app may be cluster-local or externally exposed, may implement an HTTP or a gRPC endpoint, may have been specified using built-in or custom Kubernetes resources, may use any type of horizontal or vertical autoscaling, may use any CD/GitOps process for deployment, and you may be interested in probing your cluster by injecting compute, memory, i/o, network, or any other types of chaos. Regardless of these variations, this talk will demonstrate a dead simple way to automatically launch the unified “chaos + performance validation" experiment whenever the app is updated, and automatically notify an event receiver with metrics and SLO validation results once the experiment is completed.