The Evolution and Innovation of CubeFS

Hybrid Cloud Acceleration for GPU Computing

• Self-build cloud GPU resources can meet user's normal training needs and sudden increment mental needs
• Maintaining a thermal computing power level in self-build clouds can reduce TCO
• Challenges of transmission and storage between public cloud and private cloud in hybrid cloud scenario
• Performance problem in storage during cross-regional due to network bandwidth
• Data security issue of public on public cloud or catch acceleration mechanism
• Implementation of local reconstruction code (LRC) to reduce bandwidth and I/O inquiries required for data recovery

In a hybrid cloud scenario, a company was facing challenges in transmitting and storing data between their self-build cloud and public cloud. They implemented a catch acceleration mechanism to reduce latency and improve performance, but faced data security issues on the public cloud. To address this, they implemented LRC which reduced bandwidth and I/O inquiries required for data recovery, resulting in lower storage costs and improved data durability.

CubeFS is a cloud native distributed storage solution for big data processing, machine learning，data sharing and protection, etc. CubeFS is compatible with three access protocols(S3/POSIX/HDFS)， it provides multiple data redundancy strategies ，such as replication and erasure-coding, it aslo support hybrid-cloud acceleration。 In this talk, there will be several parts to have an introduction of Cubefs and deep-dive discussions to talk about the technical details, the recent release, and future plans. CubeFS was accepted as an incubating project by the Cloud Native Computing Foundation in July 2022