Does AI Coding Actually Make Developers Faster? (It Depends)

The honest answer is: it depends — and the evidence genuinely points both ways. In mid-2025, METR ran a randomized controlled trial where 16 experienced developers were 19% slower with AI on their own mature repos, even though they predicted a 24% speedup and still felt 20% faster afterward. Meanwhile a GitHub Copilot trial found devs 55.8% faster on a greenfield task. Both are real. This is a guide to which one applies to you.

The short answer: it depends, and the data proves it

Ask the internet whether AI makes developers faster and you get two confident, opposite answers. One camp waves the METR study — experienced engineers went 19% slower with AI. The other waves adoption surveys — most developers say AI makes them more productive. Both camps are quoting real numbers. They are just measuring different worlds.

Here is the reconciliation up front, so the rest of the article earns it: AI coding tools help most on greenfield, boilerplate, unfamiliar-stack and low-stakes work, and for less-experienced developers. They can actively slow down expert work on mature, high-context, high-quality-bar codebases. The two headline trials sit at opposite ends of that spectrum — which is exactly why they do not contradict each other.

Below: four studies, a decision matrix, the perception-vs-reality mechanism, and one nuance almost every blog gets wrong — METR did not retract its finding.

The METR study: 19% slower, while feeling 20% faster

The anchor is METR's randomized controlled trial, published July 10, 2025. It is robust for what it measured. The setup:

• 16 experienced open-source developers, working on their own repositories.
• 246 real tasks (roughly 2 hours each), randomly assigned to allow or disallow AI tools.
• Repos were mature and huge — averaging 22k-plus GitHub stars and over 1M lines of code.
• Tools were the frontier of the time: primarily Cursor Pro with Claude 3.5 / 3.7 Sonnet.

The result: developers took 19% longer to complete issues when allowed to use AI. The twist that made it go viral was the perception gap. Before starting, the same developers forecast a 24% speedup. After finishing — after being measured as slower — they still believed AI had sped them up by about 20%. That is a roughly 39-point gap between what happened and what it felt like.

METR's own factor analysis pins the slowdown on context-specific causes: developers already knew these repos intimately, the codebases were large and complex, the implicit quality standard was high, the 2025 models were not always reliable, and a lot of time went into reviewing and fixing AI output — layered on top of general over-optimism about how useful AI would be.

The counter-evidence: greenfield, juniors, and surveys

Now the other end of the spectrum. The GitHub Copilot RCT (Peng et al., 2023) found the AI-assisted group finished a task 55.8% faster (95% CI 21–89%), and developers with less programming experience benefited most.

But look at the task: implement an HTTP server in JavaScript from scratch. That is a self-contained, greenfield, well-trodden problem — close to the near-opposite of editing a 1M-line repo you maintain. Different condition, valid result. It is also worth noting the authors were affiliated with GitHub / Microsoft, the tool vendor, so treat the exact figure as vendor-adjacent.

Two more data points, both self-reported:

• DORA 2025 found — unlike 2024 — a positive relationship between AI adoption and delivery throughput, but AI still had a negative relationship with delivery stability. Its framing: AI is an amplifier of existing team conditions, not an automatic improvement. Strong foundation, AI helps; weak foundation, AI magnifies the mess.
• Stack Overflow 2025: about 84% use or plan to use AI tools, and among agent users roughly 70% say agents cut time on tasks and ~69% say they raised productivity. Yet trust is falling — about 46% distrust AI accuracy versus roughly a third who trust it.

The catch, which we return to below: DORA and Stack Overflow measure perception and adoption, not stopwatch time. That distinction is the whole ballgame.

Four studies, side by side

Stacking the evidence in one table makes the pattern obvious: the two studies that measured time with a stopwatch disagree only because their conditions are opposite, and the two self-reported studies measure a different thing entirely.

The highlighted cells are the honest apples-to-apples pair: two measured trials, opposite conditions, opposite results. Neither refutes the other. And because METR's central finding is that feeling diverged from measurement, you cannot use the survey rows to wave away the METR row — they are not even measuring the same variable.

Why it feels faster than it is

The perception gap is the most useful thing in this entire literature, because it explains why smart people disagree. Generating code with AI feels fast: you type a prompt, a wall of plausible code appears, dopamine fires. The hidden cost is downstream — reading it, catching the parts that are almost-right-but-not-quite, and debugging what you did not write. Stack Overflow 2025 quantifies the friction: 45% of developers said debugging AI-generated code takes longer than writing it themselves, and 66% struggle with solutions that are almost right.

This is why adoption surveys run positive while a stopwatch runs negative. A survey captures the bright, felt part of the loop — the generation — and discounts the dim, hidden part — the review and rework. Both can be honest reports of the same experience. Only one of them is time.

When AI coding helps vs hurts: a decision rule

Map your situation across four axes. The more rows that fall on the left, the more AI is likely to speed you up; the more on the right, the more it can slow you down.

A few practical corollaries. If you are exploring a new framework, AI is a superpower — it is the greenfield/unfamiliar quadrant twice over. If you are the maintainer making a careful fix in code you wrote, be honest that generation may feel fast while review eats the savings. And structure matters: spec-driven development and a good AGENTS.md file shrink the review-and-rework tax that dragged METR's numbers down, which is also why the difference between vibe coding vs agentic coding shows up in the results.

Cost is the other lever most speed debates ignore. If AI adds review time, the model's price per task has to earn it back — see our LLM coding cost benchmark and the roundup of the best cheap LLM for coding for where the value sits. Routing boilerplate to a cheap model like Claude 3.7 Sonnet and hard reasoning to Claude Opus 4.8 is the pragmatic middle path.

The 2026 twist: METR did not retract, but can no longer measure

Here is the nuance most secondary summaries get flat wrong. A widely-circulated claim says METR retracted the 19% figure over methodological flaws. That is false. METR's February 24, 2026 update explicitly stands by the original result as a historical, context-specific finding.

What actually happened is more interesting. METR tried to run a follow-up and abandoned the experiment design — not because the first one was wrong, but because developers now refuse to work without AI. That refusal creates selection bias: you can no longer cleanly recruit a no-AI control group or assign no-AI tasks without distorting who participates and what they pick. METR suspects real-world AI impact has likely improved since early 2025, but their clean measurement approach can no longer capture it. The flaws critics cite belong to that later, separate follow-up — not the original RCT.

The takeaway is almost poetic: the field can no longer run the experiment that would settle the debate, because the thing being studied has become non-optional. So treat every number here as time-stamped to its tools. The right question is not the permanent one everyone argues about — it is the local one you can actually answer: on this task, in this codebase, at my level, with today's model — measured, not felt — am I faster?

FAQ