Dates
Conferences
Tags
Sort by:
Most recent
Conference: KubeCon + CloudNativeCon Europe 2023
Authors: Florent Poinsard, Arthur Schreiber
2023-04-21
tldr - powered by Generative AI
GitHub uses MySQL and v-test for their database management and scaling strategy
- GitHub has a standard MySQL setup with 80 clusters and 2000 instances
- They have a read-heavy load with 330 terabytes of data across primaries and replicas
- Their scaling strategy includes setting up separate clusters for new features, breaking up existing clusters, and adding more replicas
- They ran into problems with scaling approaches and schema migration times
- They implemented v-test as a solution, which is a sharding model that fits their data model well
- v-test allows for seamless schema changes, automatic failure detection and repair, and query consolidation
- GitHub has successfully migrated 20 key spaces to v-test, reducing the number of hosts needed and improving read and write rates
Conference: KubeCon + CloudNativeCon Europe 2023
Authors: Cyril Corbon, Alex Triquet
2023-04-21
tldr - powered by Generative AI
The presentation discusses the use of Kubernetes for running stateful sets and taking advantage of its features for data management, database management, application monitoring, application deployment monitoring, logging, machine learning, and university management.
- Kubernetes is used for running stateful sets and taking advantage of its features for data management, database management, application monitoring, application deployment monitoring, logging, machine learning, and university management
- The presentation discusses the use of Apache Druid for data ingestion and reconciliation
- The presentation highlights the importance of RAM for Druid clusters and the benefits of caching segments
- The presentation discusses the plan to migrate to version 325 and Java 17, decrease costs by migrating to ARM, and use the TCD and Kubernetes API for endpoint and information retrieval
- The presentation acknowledges the challenges of running stateful sets on Kubernetes but believes it is the best option
- The presentation expresses gratitude to the Druid and Druid operational community for their support
Conference: KubeCon + CloudNativeCon Europe 2023
Authors: Matt Turner, Liam White
2023-04-20
Kubernetes is a very extensible system, to the point that the apiserver and database can be run on their own. In this configuration there's no controller-manager or scheduler, and no support for actually running workloads. However these components can support CRDs and Operators. This makes it a perfect host for lightweight control planes for other systems. In this talk, Matt will show how an Operator can run on just a bare-bones control plane. The control-plane cannot run workloads and the Operator (Istio, in this case) doesn't deal with anything in the cluster. However, together they functions as a small, lightweight unit providing services outside the cluster. Matt will explain the theory of this style of deployment, and how to set it up yourself. He will show a demo using the Istio control plane, which will provide networking services to a set of VMs (as there is no cluster).
Conference: KubeCon + CloudNativeCon Europe 2022
Authors: James Zhang, Xinye Tao
2022-05-19
tldr - powered by Generative AI
TiKV has introduced new features to improve its service quality in cloud storage, including Raft Engine and Prioritized I/O Rate Limiting.
- TiKV is a distributed storage engine that can scale out to hundreds of nodes and replicate both wall and data files to provide high availability.
- Cloud storage hardware can be a challenge due to its internal complexity, higher latency, and shared hardware with other users.
- To reduce cost and improve scalability, TiKV has introduced two new features: Raft Engine and Prioritized I/O Rate Limiting.
- Raft Engine is a new log store for TiKV that maintains an email index of all log entries, reducing background works and compressing log entries with lz4 to reduce nearly 30% of all server writes.
- Prioritized I/O Rate Limiting categorizes all system I/Os into three different priorities and assigns individual I/O limits to those priorities, adjusting the I/O limits for lower priorities after an overflow to decrease the global I/O usage of the system.