Context Engineering as a Discipline: Building Governed AI Analytics

In this episode of the Data Engineering Podcast, Tobias Macey interviews Nick Schrock, CTO and founder of Dagster Labs, about Compass, an AI analytical system designed to keep data teams in the loop. Nick discusses the evolution of AI and its impact on software, infrastructure, and stakeholder relationships. He emphasizes Compass's goal of restructuring the relationship between data platform teams and stakeholders by providing a Slack-native experience for interacting with data in natural language. The conversation covers context engineering, context management, and the importance of human oversight in AI-driven analytics. They also explore the challenges and opportunities of using agentic systems, including cost management and the integration of AI into data infrastructure. Nick shares insights on the innovative applications of Compass and its potential to transform various roles within organizations.