logo

Clusterless - Utilizing a Cluster of Clusters Seamlessly

2021-10-14

Authors:   Chris Hein, Fei Guo


Summary

Clusterless is a new design that allows for the seamless utilization of clusters of clusters to run workloads wherever needed, reducing the management burden of multi-cluster users.
  • Multi-cluster support in Kubernetes is important for application efficiency, better resource utilization, and breaking geographic boundaries.
  • Existing open source projects have addressed challenges in multi-cluster management but either introduce new APIs/workflows or have scalability limitations.
  • Clusterless abstracts multiple clusters as Pod resource providers, connecting to a management cluster exposed to users, without introducing new APIs and preserving consistent user experiences.
  • Pod dispatching is done by a highly efficient two-level scheduling algorithm achieving O(n) scheduling throughput, where n is the number of managed clusters.
  • Clusterless is great for running more of what you already have in a single cluster, but not for redundancy or availability of workloads or heterogeneous hardware.
As companies scale their Kubernetes architecture, they often turn to multi-cluster management to horizontally scale their workloads and deploy them in specific regions. However, this brings in new challenges such as security compliance and the autonomy of workloads. Clusterless addresses these challenges by allowing for the seamless utilization of clusters of clusters, reducing the management burden of multi-cluster users. It abstracts multiple clusters as Pod resource providers and connects them to a management cluster exposed to users, without introducing new APIs and preserving consistent user experiences. The highly efficient two-level scheduling algorithm achieves O(n) scheduling throughput, making it great for running more of what you already have in a single cluster.

Abstract

Multi-cluster support in Kubernetes becomes more and more important for application efficiency, better resource utilization and breaking the geographic boundary. While existing open source projects like Karmada, Tensile-Kube, Liqo, Admiralty, and more have addressed challenges in multi-cluster management such as configuration, lifecycle management, they either introduce new APIs/management workflows or have scalability limitations in terms of scheduling throughput. In this talk, a new Clusterless design is introduced to reduce the management burden of multi-cluster users. Basically, multiple clusters are abstracted as Pod resource providers, connecting to a management cluster which is exposed to the users. There are no new APIs introduced hence the consistent user experiences are preserved. The Pod dispatching is done by a highly efficient two-level scheduling algorithm which can achieve O(n) scheduling throughput, where n is the number of managed clusters.

Materials: