Understanding Isolation Levels in the Kubernetes Landscape

The presentation discusses the importance of isolation and multi-tenancy in Kubernetes clusters, and provides insights on how to achieve this through atomic primitives, namespaces, and container security.

• Containers are the atomic unit of work in Kubernetes, and run on pods which are logical units of an application deployed to nodes in a cluster
• Isolation layers in Kubernetes include the workload, control plane, and platform services
• Namespaces are a key way to enforce isolation and can be defined in various ways depending on the multi-tenant architecture
• Container security is crucial in maintaining isolation, and can be achieved through vulnerability scanning and treating containers as immutable objects
• An anecdote is provided on how the University of Chicago used pod-level isolation to enable users to launch their own Jupyter notebooks while maintaining isolation from other pods

The University of Chicago used pod-level isolation to enable users to launch their own Jupyter notebooks into a browser setting for analyzing biomedical data. Hatchery, an open source tool, deployed a pod for each user with a persistent volume for storing the notebook. The pod was isolated from other pods and had constraints on CPU and memory usage, as well as denied access to the Kubernetes API and internal APIs through network policies.

In building open-source software with Kubernetes, it becomes important to understand the support and limitations for isolation and security at different levels. The ecosystem can be complex and it might be challenging to verify or fully understand the guarantees for isolation at each layer - from the cluster level to the container level. More importantly, how do you know what level of isolation you need at each level? Maybe if your cluster is secure, there is less of a concern for container level isolation? This talk will go over the impact and tradeoffs for optimizing for isolation at a given layer and help you understand what can be done at the cluster level, the namespace level, the pod, the container. As an example, this talk will present the case study of running Jupyter Notebooks within Kubernetes and supporting the requirement to provide isolation between each pod in order to create separate user spaces for each notebook launched in the cluster.