As AI becomes embedded in software development, many of the metrics that engineering organizations have relied on for years are starting to break down.

In this session from DX Annual, Uber's Ty Smith and Abhishek Tibrewal share how their approach to measuring AI's impact on developer productivity has evolved over time. They walk through the different phases of their measurement journey, from adoption and engagement to measuring impact, ROI, and agentic value, explaining what they chose to measure at each stage, what worked, what failed, and how their thinking changed along the way.

They also discuss the role of qualitative feedback before telemetry existed, the challenge of identifying meaningful engagement signals, why "developer years saved" failed as an ROI metric, and how AI agents forced them to rethink traditional productivity measurements. Finally, they introduce Uber's emerging framework built around feature velocity and explore the unanswered questions that remain as software development becomes increasingly agent-driven.

Where to find Abhishek Tibrewal

• LinkedIn: https://www.linkedin.com/in/aabhishektibrewal

Where to find Ty Smith:

• LinkedIn: https://www.linkedin.com/in/tyvsmith

In this episode, we cover:

(00:00) Intro

(01:30) Steve Yegge’s 8 stages of AI-assisted development

(03:22) Uber’s shift to a generative AI-powered company

(04:20) Uber’s pre-AI productivity metrics

(06:55) Important questions from stakeholders that previous metrics didn’t answer

(08:25) How Uber measures AI before telemetry exists

(11:11) Metrics used to measure adoption

(12:49) Measuring engagement

(14:30) Measuring impact

(16:32) The challenge of measuring AI ROI

(19:32) Rethinking adoption, engagement, and impact for agentic AI

(26:01) The new north star: Feature velocity

(28:41) PR classification + feature velocity: the questions it can answer

(33:01) What comes next and what’s still unanswered

(34:30) Lessons learned and what they'd do differently

(37:11) Q&A #1: How Uber defines a feature

(38:50) Q&A #2: Measuring success and AI ROI

Referenced:

• Welcome to Gas Town

• Dara Khosrowshahi (Uber CEO)

Uber’s journey of measuring AI impact on developer productivity

Engineering Enablement by DX

Uber’s journey of measuring AI impact on developer productivity