Can you prove AI ROI in Software Eng? (Stanford 120k Devs Study) – Yegor Denisov-Blanch, Stanford

Yegor Denisov-Blanch discusses research on the impact of AI on software engineering productivity, highlighting a widening gap between top and bottom performers. The talk covers factors driving AI productivity gains, an AI practices benchmark, measuring AI return on investment, and a case study. Key points include the importance of code-based hygiene, managing code base entropy, and understanding when to use AI. The presentation introduces a framework for measuring ROI using a primary metric (engineering output) and guardrail metrics (rework, quality, people & DevOps). A case study illustrates the pitfalls of solely measuring pull requests, emphasizing the need for thorough evaluation to understand AI's true impact. The talk concludes with an invitation to participate in the research for more personalized insights.