The discussion centers on the growing importance of orchestration in AI and data pipelines, particularly context engineering. Nick Schrock, founder and CTO of Dagster, distinguishes between dynamic orchestration for agentic workflows and data pipeline orchestration, which Dagster focuses on. He emphasizes that a solid data strategy is foundational for any AI initiative, highlighting context pipelines as the new data pipelines. Schrock introduces Compass, Dagster's exploratory data analysis tool, which uses a Slack-based interface to streamline data querying and context management. The goal is to enable business stakeholders to contribute to a governed "context store," improving the accuracy and relevance of data-driven insights. Schrock also shares his enthusiasm for AI-accelerated software engineering, noting its transformative impact on coding practices.
Outlines
Part 1: AI Orchestration and Data Strategy
Part 2: Introducing Compass and the Context Store
Part 3: Implementation and User Experience
Part 4: Workflows and Platform Integration
Part 5: The Future of AI Engineering
Sign in to continue reading, translating and more.