Find Out About The Technology Behind The Latest PFAD In Analytical Database Development

This podcast episode explores the journey of Paul Dix, the founder and CTO of InfluxData, and his development of InfluxDB, an open source time series database. The episode discusses the design goals and constraints of InfluxDB 3.0, the motivations behind the redesign of InfluxDB 2.0, the challenges of building a fully executable database engine, the benefits of building a data platform on open source components, the integration of InfluxDB 3 with Amazon query services, the advantages of adopting open source platform pieces for startups, the challenges of building a data system using data processing libraries, the potential of Rust in big data processing, the landscape of columnar serialization formats, and the future of data processing. Overall, the episode highlights the importance of open source development and the potential for innovation in the data processing space.

Outlines

Sign in to continue reading, translating and more.

Continue

Data Engineering Podcast

Paul Dix's Journey into Data Engineering and the Apache Arrow Ecosystem

InfluxDB 3.0: Achieving Infinite Cardinality and Ecosystem Compatibility

InfluxDB 2.0: Rethinking the Architecture for Infinite Cardinality and Broader Ecosystem Compatibility

The Challenges of Building a Database Engine on Disparate Parts

Apache Iceberg and Flight SQL: Enhancing Data Platform Integrations

Arrow Project and ODBC: New Standards for Data Connectivity

The Open Source Advantage: Outpacing Closed Source Development

The Scaffolding Around Data Processing Libraries

Rust's Potential to Revolutionize Big Data Processing

Parquet's Dominance and Surprises in the Data Ecosystem

The Future of Data Processing: Bridging the Gap Between Warehousing and Streaming

Bridging the Gap between Stream Processing and Data Warehousing

Find Out About The Technology Behind The Latest PFAD In Analytical Database Development

Data Engineering Podcast

00:11Paul Dix's Journey into Data Engineering and the Apache Arrow Ecosystem

Paul Dix's Journey into Data Engineering and the Apache Arrow Ecosystem

04:35InfluxDB 3.0: Achieving Infinite Cardinality and Ecosystem Compatibility

InfluxDB 3.0: Achieving Infinite Cardinality and Ecosystem Compatibility

07:37InfluxDB 2.0: Rethinking the Architecture for Infinite Cardinality and Broader Ecosystem Compatibility

InfluxDB 2.0: Rethinking the Architecture for Infinite Cardinality and Broader Ecosystem Compatibility

13:08The Challenges of Building a Database Engine on Disparate Parts

The Challenges of Building a Database Engine on Disparate Parts

17:04Apache Iceberg and Flight SQL: Enhancing Data Platform Integrations

Apache Iceberg and Flight SQL: Enhancing Data Platform Integrations

22:08Arrow Project and ODBC: New Standards for Data Connectivity

Arrow Project and ODBC: New Standards for Data Connectivity

27:51The Open Source Advantage: Outpacing Closed Source Development

The Open Source Advantage: Outpacing Closed Source Development

32:15The Scaffolding Around Data Processing Libraries

The Scaffolding Around Data Processing Libraries

36:14Rust's Potential to Revolutionize Big Data Processing

Rust's Potential to Revolutionize Big Data Processing

42:13Parquet's Dominance and Surprises in the Data Ecosystem

Parquet's Dominance and Surprises in the Data Ecosystem

48:55The Future of Data Processing: Bridging the Gap Between Warehousing and Streaming

The Future of Data Processing: Bridging the Gap Between Warehousing and Streaming

53:19Bridging the Gap between Stream Processing and Data Warehousing

Bridging the Gap between Stream Processing and Data Warehousing