Multi-Tenancy in Kubernetes: How We Avoided Clusters Sprawl With Capsule

Capsule is a multi-tenancy framework that allows users to create and manage tenants and known groups in an internal cluster, providing additional services and integrations with internal systems. Capsule helps implement a bring your own device approach, provides additional service abilities, and ensures isolation between tenants. Capsule proxy modifies user's requests so they only see resources available in their tenant, providing users an experience that they are working in their own private cluster. Tenant after scalar is a forked version of cluster after scalar which watches for unschedulable ports only in the tenant namespaces.

• Capsule is a multi-tenancy framework that allows users to create and manage tenants and known groups in an internal cluster
• Capsule helps implement a bring your own device approach
• Capsule provides additional service abilities
• Capsule ensures isolation between tenants
• Capsule proxy modifies user's requests so they only see resources available in their tenant
• Tenant after scalar is a forked version of cluster after scalar which watches for unschedulable ports only in the tenant namespaces

Capsule had an ability to assign specific labels or annotations on tenant namespaces and tenant services so namespace labels are used in order to achieve network resolution with kaliko global network policies and our tenants are always isolated from each other. With the help of labels on services, we are able to make notepods to be opened only on the tenant nodes.

Kubernetes is great when you have to deal with it as is and plays well with isolating workloads and limiting resources due to its primitives as ResourceQuota, LimitRange, NetworkPolicy, Namespace, and so on. But is it enough? Can you create a real isolated multi-tenant environment in it? With our experience with many production environments, we discovered it's not entirely. That is the reason why Capsule, an open-source Kubernetes Operator for multi-tenancy was born. Expanding the authentication Kubernetes capabilities, Capsule provides a viable and robust solution to avoid the hyped cluster sprawl while maintaining a native Kubernetes UX. And, last but not least, allowing a BYOD to push compute, storage, and network isolation and avoiding the noisy neighbors' effect. After a brief overview of the project by Dario Tranchitella (maintainer), Max Fedotov will explain how Capsule enhanced the operations and supercharged the provisioning mechanism for their k8s cluster at Wargaming.net