185: The Evolution of Data Processing, Data Formats, and Data Sharing with Ryan Blue of Tabular

This week on The Data Stack Show, Eric and Kostas chat with Ryan Blue, the Co-Founder and CEO of Tabular, and also creator of Iceberg and former Cloudera and Netflix employee. During the episode, Ryan discusses the challenges of managing large-scale data and the development of Iceberg, a new table format. He explains Iceberg's benefits, such as automatic partitioning and improved metadata management, which simplify data engineers' tasks and enhance query performance. The conversation covers the importance of atomicity in analytics systems, the scalability of Iceberg, and the trade-offs in mixed workload environments. Additionally, Ryan addresses the differences in cloud object storage performance and the integration of security and access controls into distributed file systems. He also touches on recent Iceberg updates, including Python and Rust support, the anticipation of view support in the upcoming release, and more.