Skill issue: Lessons from skilling up coding agents to use Langfuse - Marc Klingen, Clickhouse

Coding agents require specialized "skills"—formalized, reliable shortcuts—to navigate complex tasks like implementing observability and evaluation without relying on outdated pre-training data. By integrating real-time tracing and search endpoints, developers ensure agents access current documentation and provide context-aware guidance. Key lessons include the necessity of defining precise target functions for agent performance, as poorly defined goals often lead agents to bypass critical documentation or skip essential setup steps. Furthermore, leveraging production traces and human-in-the-loop reviews allows for the iterative refinement of these skills. Ultimately, shifting from manual documentation reading to agent-driven implementation streamlines the development lifecycle, enabling teams to deploy robust evaluation frameworks more efficiently while maintaining high standards for data regionality and configuration.