Graduated Project Lightning Talk: Time to Live (TTL) Support for TiKV

TiKV achieved more flexible TTL support by leveraging the hook of RocksDB’s compaction filter and user defined properties, and made the disk space of TTL key-values recycled in time.

• TiKV is a distributed key value database built on RocksDB with Raft
• RocksDB supports TTL natively but with a limitation that all keys should be of the same TTL
• TiKV achieved more flexible TTL support by appending TTL information as 8 bytes to the value when writing TTL
• TiKV leveraged the hook of RocksDB’s compaction filter and user defined properties to make the disk space of TTL key-values recycled in time
• TiKV provides horizontal stability and high availability based on Raft and supports both optimistic and pessimistic distributed transaction

TiKV team faced the issue of inconvenience for developers who may have the demand to set different TTL for each key. To solve it, they supported TTL in TiKV level instead of using RocksDB’s TTL. They achieved this by leveraging the hook of RocksDB’s compaction filter and user defined properties, and made the disk space of TTL key-values recycled in time.

Time to live (TTL) is a beneficial feature for developers to invalidate keys automatically. RocksDB supports TTL natively but with a limitation that all keys should be of the same TTL. This may cause inconvenience for developers who may have the demand to set different TTL for each key. This issue happens to TiKV, a distributed key value database built on RocksDB with Raft. To solve it, TiKV team achieved more flexible TTL support in TiKV level instead of using RocksDB’s TTL. In this talk, Bokang Zhang will walk through the TiKV architecture and talk about how they supported TTL in TiKV by leveraging the hook of RocksDB’s compaction filter and user defined properties, and how they made the disk space of TTL key-values recycled in time.