Evolving MLOps Platforms for Generative AI and Agents with Abhijit Bose - #714

The discussion centers on Capital One's AI platform strategy, particularly regarding the integration of GenAI with traditional machine learning operations. Abhijit Bose, Head of Enterprise AI and ML Platforms at Capital One, details the company's platform-centric approach, emphasizing the importance of centralized governance, resource optimization, and a strong data culture. A key focus is on extending their existing MLOps platform to accommodate GenAI, addressing new challenges like LLM hallucinations and complex agentic workflows, and building infrastructure for fine-tuning models. Bose highlights a customer service use case where fine-tuning Llama models improved agent efficiency by enhancing search and summarization accuracy, and he also shares insights into how Capital One approaches model selection, fine-tuning, and inference optimization.