What's the Future of Data Engineering in an AI World? S3E19

Data engineering in asset management is undergoing a fundamental shift from building rigid, upfront infrastructure to leveraging AI for real-time inference. Instead of investing months in deterministic data warehouses, firms should prioritize a dynamic "context layer" that captures business logic, taxonomy, and methodology in structured formats like markdown. This approach allows AI agents to perform complex data transformations and analysis on the fly, significantly reducing time-to-value. Success in this paradigm requires tight alignment between the context layer, agent harnesses, and user interfaces to ensure data integrity and observability. By pushing the contextualization problem to the last mile of inference, organizations can iterate faster, reduce reliance on inefficient human-to-human communication, and maintain control through robust permissioning and clear accountability in AI-driven workflows.