How to measure AI developer productivity in 2025 | Nicole Forsgren

Overview

This episode dives into the evolving landscape of developer productivity in the age of AI, featuring insights from Nicole Forsgren, a leading expert in developer experience (DevEx). The discussion explores the challenges of measuring productivity, the impact of AI on engineering workflows, and practical strategies for improving developer experience. Nicole also introduces her new book, Frictionless, which outlines a seven-step framework for building and scaling DevEx teams.

Notable Quotes

- Most productivity metrics are a lie. If the goal is more lines of code, I can prompt something to write the longest piece of code ever. It's just too easy to game that system. — Nicole Forsgren, on the pitfalls of traditional productivity metrics.

- We can ship trash faster every single day. We need strategy and really smart decisions to know what to ship. — Nicole Forsgren, on the importance of aligning speed with meaningful outcomes.

- Now we can also make a 45-minute work block useful because getting into the flow is actually kind of handed off, at least in part, to the machine. — Nicole Forsgren, on how AI is reshaping workflows and flow states.

🛠️ Developer Experience (DevEx) and Productivity Metrics

- Nicole Forsgren explains DevEx as the day-to-day experience of building software, including friction, workflows, and support systems.

- Poor DevEx can undermine even the best tools and processes, leading to burnout and reduced innovation.

- Traditional productivity metrics like lines of code are outdated and easily manipulated, especially with AI tools generating verbose code.

- New metrics should focus on outcomes like code survivability, quality, and feedback loops rather than raw output.

🧠 Flow State and Cognitive Load in the Age of AI

- AI is changing how developers achieve flow states, shifting focus from writing code to reviewing and coordinating AI-generated outputs.

- While AI can disrupt traditional flow, it also offers opportunities to rethink workflows and make shorter work blocks more productive.

- Tools like AI agents can assist developers in maintaining context, generating diagrams, and reducing cognitive load.

📈 Measuring Productivity Gains from AI

- Companies often struggle to measure the impact of AI tools on productivity. Nicole Forsgren advises focusing on metrics that align with leadership priorities, such as:

- Time from feature idea to production or experimentation.

- Cost savings from reduced toil, fewer failed tests, or vendor spend.

- Velocity improvements across the development pipeline.

- Surveys can provide quick insights into developer satisfaction and barriers to productivity.

📚 Building and Scaling a Developer Experience Team

- Nicole Forsgren introduces her seven-step framework from her upcoming book, Frictionless:

1. Start the journey with a listening tour to identify pain points.

2. Achieve quick wins to build momentum.

3. Use data to optimize workflows and identify priorities.

4. Decide on strategy and prioritize initiatives.

5. Sell the strategy to stakeholders and gain buy-in.

6. Drive change at scale, whether locally or globally.

7. Evaluate progress and demonstrate value.

- The framework is designed to be adaptable, allowing teams to start at any step based on their current stage.

🚀 Real-World Impact of AI on Developer Productivity

- AI tools like OpenAI Codex and Copilot are enabling rapid prototyping, bug detection, and automated unit test generation.

- AI can unblock developers, accelerate workflows, and improve documentation quality, but its impact is difficult to measure directly.

- Combining AI tools with improved DevEx can lead to significant productivity gains, but attribution challenges remain.

- Companies should focus on aligning AI adoption with strategic goals and measuring outcomes that matter most to their leadership.