Data Engineering Podcast

Reducing Data Debt with Agile Ledger Architecture

What Context Really Means in Data Engineering and AI

Context engineering has become a critical discipline for ensuring trustworthy AI outcomes as agents replace human analysts as the primary consumers of organizational data. Defining context requires integrating formal structures like semantic layers and data catalogs with informal tribal knowledge, all while maintaining...

Specialized AI for Data Engineers: Inside Astronomer’s Otto

Why Multi-Agent Systems Need Shared State, Graph Semantics, and Governance

OmniGraph functions as a Lakehouse-native graph engine designed to serve as a foundational substrate for multi-agent systems. By integrating Git-style semantics—such as branching and merging—into the graph storage layer, the system provides a governance mechanism for probabilistic agents that act as autonomous writers....

Building the Context Flywheel for AI Data Agents

Building a context flywheel for AI agents requires moving beyond traditional metadata catalogs toward a comprehensive context layer that integrates institutional, semantic, and procedural knowledge. Prukalpa Sankar, founder of Atlan, emphasizes that while AI models have achieved exponential intelligence growth, their e...

Holding Kafka Right: Product-Friendly Streaming with TypeStream

Text to Data Products: Kaarvi’s End-to-End AI for Ingestion, Quality, and Dashboards

Scaling Graph Analytics Without ETL: Inside PuppyGraph’s Architecture

Maximizing GPU Utilization: Heterogeneous Pipelines with Ray and Kubernetes

The AI-First Data Engineer: 10–50x Productivity and What Changes Next

Agentic coding—AI systems capable of autonomously executing tasks, validating results, and iterating toward a goal—is fundamentally transforming data engineering. Moving beyond simple code generation, this shift allows data professionals to act as drivers of autonomous processes, significantly increasing productivity a...

Treat Metering Like Finance: Building Data Platforms for Consumption Economics

Beyond the PDF: Rowan Cockett on Reproducible, Composable Science

Beyond Prompts: Practical Paths to Self‑Improving AI

Orion at Gravity: Trustworthy AI Analysts for the Enterprise

From Models to Momentum: Uniting Architects and Engineers with ER/Studio

From Data Models to Mind Models: Designing AI Memory at Scale

The podcast explores agentic memory architectures, focusing on how AI agents can retain state and context across interactions. Vasilije Markovich, founder of Cognee, discusses the evolution of memory solutions from simple vector stores to complex graph-vector layers that support business rules and ontologies. He detail...

Prompt Management, Tracing, and Evals: The New Table Stakes for GenAI Ops

The podcast explores the operational challenges and blind spots teams face when deploying AI models, particularly LLMs, in production. Aman Agarwal, builder of OpenLit, an AI engineering tool, highlights key issues such as understanding AI response mechanisms, managing token usage costs, and prompt management. Agarwal ...

From Legacy to AI-Ready: How MongoDB AMP Accelerates Modernization

Branches, Diffs, and SQL: How Dolt Powers Agentic Workflows

Logical First, Physical Second: A Pragmatic Path to Trusted Data

The podcast explores the critical role of data architecture in data engineering, emphasizing the need for a business-driven, logical foundation to ensure scalability and trust. Jamie Knowles, product director for ER Studio at Idera, shares his experience on how a well-defined data architecture, starting with a semantic...