One VTOrc To Rule Them All – High Availability In a Distributed Database System

The presentation discusses the engineering approach taken by Vitess to solve the consensus problem in a high QPS environment while prioritizing performance over theoretical correctness.

• Vitess is a single leader system that relies on a topology server for persistent state and recovery
• The system prioritizes performance over theoretical correctness
• Durability policy is defined as avoiding data loss and is configurable based on trade-offs between durability and availability
• Leader election has three stages: revocation, choosing a new leader, and propagation
• Planned and unplanned leader elections have different revocation processes

Vitess was able to handle millions of queries per second for reads and writes while respecting the durability policy chosen by the user. In a planned leader election, the current leader is asked to step down before a new leader is chosen, while in an unplanned leader election, a sufficient number of followers must be reached to revoke the leader. The system prioritizes performance over theoretical correctness, but still ensures that completed requests are propagated to all replicas to avoid data loss.

Vitess is a scalable, highly available distributed database system built around MySQL. It achieves scalability through sharding and durability through replication. High availability is accomplished through a Vitess feature known as cluster management. The next generation cluster management service in Vitess is called VTOrc. Users can specify their durability rules as a system configuration, which is respected while performing planned failovers. VTOrc also performs failure detection with automatic failovers while honoring the durability rules. VTOrc is already running successfully in production in multiple deployments including at PlanetScale, and it will be Generally Available in Vitess release 15 (October 25). The session will provide an introduction to VTOrc and an outline of the theory that underpins its implementation, followed by a demo of its capabilities showing multiple failover scenarios.