Data Engineering Podcast

High Performance And Low Overhead Graphs With KuzuDB

Bridging Data and Decision-Making: AI's Role in Modern Analytics

In this episode of the Data Engineering Podcast, Tobias Macey interviews Lucas Thelosen and Drew Gilson about building an autonomous data analyst. They discuss their backgrounds, how they got into data, and the inefficiencies they observed in data analytics across various companies. They introduce Orion, an AI-powered ...

From Bits to Tables: The Evolution of S3 Storage

In this episode of the Data Engineering Podcast, Tobias Macey interviews Andy Warfield about S3 Tables and S3 Vectors, exploring their functionalities and how they fit into the modern data stack. Andy discusses the evolution of S3 from a simple object storage to a more versatile system supporting advanced data types li...

Revolutionizing Python Notebooks with Marimo

Warehouse Native Incremental Data Processing With Dynamic Tables And Delayed View Semantics

In this episode of the Data Engineering Podcast, Tobias Macey interviews Dan Sotolongo, a principal engineer at Snowflake, about the challenges of incremental data processing in warehouse environments and how delayed view semantics help address the problem. Dan defines incremental data processing as efficiently updatin...

Streamlining Data Pipelines with MCP Servers and Vector Engines

In this episode of the Data Engineering Podcast, Tobias Macey interviews Kacper Lukawski, a Senior Developer Advocate at Qdrant, about using MCP servers with vector databases to streamline unstructured data processing. They discuss the challenges teams face in building pipelines for unstructured data, the applications ...

Foundational Data Engineering At 2Sigma

In this episode of the Data Engineering Podcast, Tobias Macey interviews Effie Baram, a leader in foundational data engineering at Two Sigma, about data engineering in the finance sector. Effie discusses her background in reliability engineering and how she transitioned to data engineering, highlighting the complexitie...

Enabling Agents In The Enterprise With A Platform Approach

Dagster's New Era: Modularizing Data Transformation in the Age of AI

AI and the Lakehouse: How Starburst is Pioneering New Workflows

Amazon S3: The Backbone of Modern Data Systems

Scaling Data Operations With Platform Engineering

From Data Discovery to AI: The Evolution of Semantic Layers

Balancing Off-the-Shelf and Custom Solutions in Data Engineering

StarRocks: Bridging Lakehouse and OLAP for High-Performance Analytics

In this episode of the Data Engineering Podcast, Tobias Macey interviews Sida Shen about StarRocks, a high-performance analytical database designed to support both shared-nothing and shared-data architectures. Sida explains how StarRocks evolved from Doris to address performance challenges and the need for on-the-fly q...

Exploring NATS: A Multi-Paradigm Connectivity Layer for Distributed Applications

Advanced Lakehouse Management With The LakeKeeper Iceberg REST Catalog

Simplifying Data Pipelines with Durable Execution

Overcoming Redis Limitations: The Dragonfly DB Approach

Bringing AI Into The Inner Loop of Data Engineering With Ascend

This episode explores the integration of AI into data engineering platforms, specifically focusing on Ascend.io's approach. The discussion begins by highlighting the challenges of data migrations, which are often lengthy and resource-intensive, before introducing Datafold's AI-powered solution as a contrasting example...