Conference:  OpenAI + Data Forum 2022

Authors: Ismaël Mejía

2022-06-22

Table Formats like Delta Lake and Apache Iceberg are recent storage specifications to handle slow-changing collections of files in distributed systems. They are rapidly gaining adoption by bringing new superpowers to the data engineering toolkit. In this talk, Ismaël will introduce and explain how table formats work and how features like versioning, schema evolution, time travel, and scalable metadata have positive consequences on many of the systems of the Data+AI ecosystem. From scalable metadata handling to incremental and faster data updates as well as reproducible data for AI training and inference.