Adapting TiKV for Cloud Storage
Conference: KubeCon + CloudNativeCon Europe 2022
2022-05-19
Authors: James Zhang, Xinye Tao
Summary
TiKV has introduced new features to improve its service quality in cloud storage, including Raft Engine and Prioritized I/O Rate Limiting.
- TiKV is a distributed storage engine that can scale out to hundreds of nodes and replicate both wall and data files to provide high availability.
- Cloud storage hardware can be a challenge due to its internal complexity, higher latency, and shared hardware with other users.
- To reduce cost and improve scalability, TiKV has introduced two new features: Raft Engine and Prioritized I/O Rate Limiting.
- Raft Engine is a new log store for TiKV that maintains an email index of all log entries, reducing background works and compressing log entries with lz4 to reduce nearly 30% of all server writes.
- Prioritized I/O Rate Limiting categorizes all system I/Os into three different priorities and assigns individual I/O limits to those priorities, adjusting the I/O limits for lower priorities after an overflow to decrease the global I/O usage of the system.
Abstract
TiKV is a cloud-native key-value database built in Rust. As a distributed storage layer, the underlying storage hardware plays a key role in how it performs. This session will start with an introduction to the modern cloud storage stack, highlighting the challenges and opportunities that come with it. After that, we will deep dive into several new features aimed at improving TiKV's service quality in the cloud, including Raft Engine, Prioritized I/O Rate Limiting and [...]. Finally, we'd love to get feedback as we lay out the future plan on bringing TiKV even closer to the cloud.Click here to view captioning/translation in the MeetingPlay platform!
Materials: