Agent Optimization with Pydantic AI: GEPA, Evals, Feedback Loops — Samuel Colvin, Pydantic

Optimizing AI agent performance requires a combination of structured evaluation, genetic algorithms, and dynamic configuration. The GEPA library automates prompt engineering by evolving prompts based on performance against golden datasets, effectively navigating the Pareto frontier to balance quality and cost. Integrating these agents with observability platforms like Logfire enables managed variables, allowing developers to update system prompts, model parameters, and temperature settings in production without redeploying code. While large-scale optimization is particularly valuable for private datasets where models lack inherent training, the process relies heavily on robust evaluation harnesses to define success. By treating prompt engineering as an iterative, data-driven optimization problem rather than a static task, developers can significantly improve agent accuracy and efficiency, even when utilizing smaller, faster models for complex, domain-specific tasks.