Autoscaling Elastic Kubernetes Infrastructure for Stateful Applications Using Proxyless gRPC and Istio

Kubernetes is suitable for stateless services and one of its benefits is seamless autoscaling of infrastructure in response to varying load. The resulting elasticity enables users to optimize their infrastructure and minimize costs. But what do you do if your application is stateful - for example it requires maintaining stateful sessions between clients and servers and you are using a service mesh? In this talk we will cover stateful applications where clients can create and maintain persistent sessions in the presence of load balancing where Istio routes individual RPCs to various backends. Client creates a persistent session and expects all RPCs in that session to go to a particular backend because the backend has the “state” related to that session and is not able to share the state with other backends. Note that consistent hashing or rendezvous load balancing don’t quite work because session persistence is broken when the set of backends changes. The feature uses HTTP Cookies where persistent or stateful sessions are achieved by communicating with the load balancer via cookies in Istio and proxyless gRPC. We also cover the use-case of “draining backends” where backends are gradually removed as part of downsizing the infrastructure but without breaking session persistence.