Building AI Systems You Can Trust

In this episode of the a16z AI podcast, Derrick Harris and Matt Bornstein interview Scott Clark, co-founder and CEO of Distributional, about deploying and testing AI systems, especially LLMs, in enterprise settings. The discussion revolves around establishing trust in AI for enterprise adoption, controlling usage, monitoring, and testing AI systems to track behaviors and prevent unexpected impacts from changes. Scott shares his background, highlighting the shift from focusing on performance optimization to prioritizing trust and reliability in AI systems. He defines machine learning and AI, emphasizing the generative aspect of AI and its potential value for enterprises. The conversation covers the challenges of non-deterministic and non-stationary AI systems, the importance of behavior alongside performance, and the need for centralized GenAI platforms to mitigate risks and scale AI deployments effectively.