In this monologue podcast, David from LangChain explains how to build and evaluate a customer support agent using LangGraph and LangGraph Studio. He details the agent's architecture, which includes question answering and refund subgraphs, guided by a supervisor node. David emphasizes the complexities of agent evaluation due to dynamically determined steps by LLMs, highlighting the importance of assessing both output quality and the efficiency of the agent's process. He introduces three evaluation strategies using the LangSmith SDK: evaluating final output accuracy, ensuring correct routing by the supervisor node, and verifying the agent follows an optimal trajectory. The podcast walks through setting up a golden dataset, defining application logic, and using evaluators to test the agent, providing examples and demonstrating the process within LangSmith's UI.
