Beyond the Chatbot: What Actually Works in Enterprise AI

In this episode of The Data Exchange podcast, Ben Lorica interviews Jay Alammar, Director and Engineering Fellow at Cohere, about the evolution and practical applications of Large Language Models (LLMs) in enterprise settings. Alammar discusses the shift from initial hype around generative AI to more pragmatic uses like summarization, extraction, and Retrieval Augmented Generation (RAG). The conversation covers key lessons learned by companies adopting LLMs, including the importance of data security, privacy, and the development of internal evaluation processes. They explore the evolution of RAG systems, the challenges of multimodality, and the increasing role of AI in software development, including the use of smaller, more efficient models and AI agents. Alammar also touches on emerging trends like text diffusion models and emphasizes the importance of empowering end-users with transparent AI systems.