Kubernetes Persistent Data Challenges – AZ, Region and Multi-Cloud Patterns

The presentation discusses how to complement the data layer in Kubernetes with the Container Storage Interface (CSI) plug-in and the application, using a demo of a single cluster, single availability zone, and single region.

• The demo includes two applications: Prometheus and Cassandra
• The CSI plug-in adds intelligence to the storage layer, allowing for features such as storage replication, encryption, and topology awareness
• The presentation emphasizes the importance of complementing the data layer without changing the Kubernetes architectural patterns
• The demo shows how to allow Cassandra to handle storage layer intelligence while working in harmony with the CSI plug-in and the application
• The presentation suggests that building virtual data centers in Kubernetes is a logical progression in virtualization, allowing for more agility and bespoke tools

The presenter uses AWS and i3en instances with local SSDs to achieve 300,000 IOPS, and demonstrates how to spin up a cluster using eksctl. The demo includes a job running NoSQL Bench against a Cassandra cluster, and metrics are shown for Prometheus. The presenter also mentions the importance of not stretching Kubernetes clusters across regions and instead doing so at the application level.

In this talk we are going to explore the patterns for building resilient applications with data persistence requirements on Kubernetes. We will be discussing deploying stateful workloads like Cassandra in Kubernetes using operators, with demos and examples at each stage using native objects like PVCs, dynamic provisioning and the k8ssandra operator. We will also be providing a sneak peek of some of the multi-cluster challenges with Kubernetes and Kube-Fed. Four architectural patterns will be covered with worked examples: The first pattern will be within an AZ, how to work with PVs. The next pattern will be spanning three AZs, and how to embrace or work around PV/AZ affinity constraints. The next pattern is three AZs and two regions within a single provider. Here we start to see physical limitations such as the speed of light begin to impinge on our patterns. Lastly, we move to two providers and hybrid-cloud patterns, leveraging operators to deliver our sample Cassandra workload.Click here to view captioning/translation in the MeetingPlay platform!