The Machine That Got Faster
A plant manager installs a new machine on the line. It runs twice as fast as the one it replaced, and the dashboard beside it shows the improvement on day one. Output from that station doubles, and everyone who walks past can see it.
A month later, the plant has not shipped any more product. The inspection station after the new machine is the same size it always was, run by the same people on the same shift. The parts the faster machine produces are stacked in front of it, waiting.
Nothing is broken. The new machine works exactly as promised. The plant sped up one step of a process whose pace was set somewhere else, and then reported on the step it sped up.
That is where many engineering organizations now stand with AI coding tools.
The Number Everyone Is Quoting
A 2026 Harvard working paper studied what happens inside companies after they adopt AI coding agents. The data came from an engineering analytics platform and covered roughly 300 million work events across 718 firms, from January 2021 to March 2026. After adoption, lines of code rose about 30%, commits about 20% and pull requests about 23%.
Then the authors looked at finished work: issues and larger features (epics) resolved. The effect was small and not statistically significant. Their estimates rule out an increase in output of more than 12% of the baseline, well short of the 30% rise in code. More code did not pass through to more software.
When the story reached Hacker News under the headline "AI coding agents generate more code, but not more software," it collected 364 points and 313 comments. The argument in the comments is the one engineering leaders are having in their own leadership meetings. Some said review has become the bottleneck and the work is shifting to specs, tests and verification. Others said review is the thing that keeps a team's shared understanding alive, and the last place to cut corners.
Both sides are describing the same thing. The tools made one stage of software delivery faster. Nobody redesigned the rest of the pipeline. And most organizations are still measuring the stage that got faster.
The Wrong Diagnosis
When AI adoption does not show up in delivery, the usual conclusion is that adoption is not deep enough. Teams are not using the tools properly. The licenses are underused. The answer is more training, better prompts, a mandate to use the agent for every ticket.
Sometimes that is true. Usually it is not.
The same study found that engineers spend only 30 to 40% of their time writing code. The rest goes to planning, scoping, reviewing, testing and coordinating. Make the coding part faster and you have sped up a third of the job. And the extra code does not disappear. It lands on the stages that did not get faster.
That is not an adoption problem. It is a constraint problem wearing an adoption problem's clothes.
The cost shows up in three places:
Volume mistaken for value. Lines, commits and pull requests are the easiest numbers to put on a slide. Leadership reports a gain no customer ever sees.
Review becomes the queue. After agent adoption, the time to review a pull request rose 49%, the share sent back for changes nearly doubled, and comments per pull request rose 35%.
Decisions made on the wrong number. Budgets, hiring plans and roadmap promises get set on keyboard metrics, and inherit whatever is wrong with them.
Find the Station That Is Actually Slow
None of this is new. Eli Goldratt described it decades ago in The Goal: a system moves only as fast as its slowest step, and improving any other step just builds inventory in front of the constraint. That is the plant from the opening, and the pile of parts in front of its inspection station.
AI coding tools are that faster machine. In the study, the inspection station was code review. In your organization it may also be test environments, a release process or the specification work that happens before anyone writes a line. The work in progress in front of them is the pile of pull requests waiting for a human.
%% caption: AI speeds up one stage; the work waits at the next
flowchart LR
A[Plan and specify] --> B[Write code
AI makes this faster]
B --> C[[Wait for review
work piles up here]]
C --> D[Review and rework]
D --> E[Test environments]
E --> F[Release to production]
F --> G[Software users
actually get]
So the first job is not to measure AI. It is to find where work waits. Trace a sample of recent changes from the moment work was picked up to the moment it reached production, and note how long each spent being written, waiting for review, in review, waiting for an environment and waiting for release. The writing stage is usually not where the time goes, and AI makes it shorter still.
This is the part of developer experience I spent the most time on as Manager of Development Experience at Spark Networks: tooling, environment reliability and the delivery friction that never shows up on a sprint board. Friction you have not traced is friction you are guessing about. I wrote about treating those conditions as a delivery control system rather than a perk. AI does not change that lesson. It moves the friction.
Measure the System, Not the Keyboard
Once you know where work waits, the measurement question becomes much simpler. You need a small number of signals that describe the whole pipeline, not the stage the tools touch. The DORA metrics are the obvious starting point and most of what follows builds on them, with two additions that matter specifically for AI-assisted work.
| Instead of measuring | Measure | What it tells you |
|---|---|---|
| Lines of code, commits | Work items finished and deployed per period | Whether more software is reaching users, which is the only output that counts |
| Pull requests opened | Lead time from start of work to production | Whether faster writing is shortening the whole journey or just one leg of it |
| AI tool usage and acceptance rates | Time waiting for review, and time in review | Where the extra code is piling up |
| Suggestions accepted | Share of changes sent back for rework | Whether AI-written code arrives ready, or needs a second pass |
| Developer hours "saved" | Change failure rate and time to restore | Whether speed is being bought with quality, security or reliability |
Two cautions. First, never turn any of these into an individual target. The moment a team's lead time becomes a person's bonus, the number stops describing the system. Second, keep the scorecard short. Five numbers that a leadership team reads every fortnight beat twenty that live in a dashboard nobody opens. I made the broader case for scoring things instead of describing them in you cannot improve an architecture you have not measured. The same principle applies here, aimed at delivery rather than architecture maturity.
Watch the rework signal most closely. If it rises after AI adoption, the problem is not reviewer speed. The code is arriving without something the reviewer needs.
Make Review a Designed Stage, Not a Queue
If review is the constraint, the instinct is to automate it away. The study suggests that is not enough: the bottleneck persisted after firms adopted AI code review tools, and humans still did most of the reviewing. Review is slow because every change gets the same treatment, and because reviewers must reconstruct a change's purpose from its diff.
Three changes make review a stage you designed instead of a queue you inherited.
Tier the scrutiny by risk. Not every change deserves the same review. A change to a customer-facing payment path and a change to an internal report do not carry the same risk, so they should not wait in the same line. Decide the tiers in advance, write them down, and let low-risk changes move with lighter review and stronger automated checks.
Make the purpose arrive with the change. A reviewer who can see what a component is for, what must stay true about it, and who owns it can check a change against that statement in minutes. A reviewer who cannot has to do archaeology. That is the argument I made about the intent your coding agents never see, and it is the most direct fix for a rising rework rate.
Move the rules that never change into the pipeline. A constraint that is checked by a machine on every change does not need a human to remember it. That frees reviewers for the judgment calls only they can make. I have argued for replacing the review board with automated checks for structural rules. The same logic applies to code review.
None of this needs to be heavy. In a fractional CTO engagement with a pre-seed HealthTech startup, the architecture review we set up was asynchronous with a 24-hour turnaround, and the compliance checklist lived in the pull request review rather than in a separate gate. The team shipped daily before the engagement and still shipped daily after it, now with an auditable trail. Designed review is faster than review nobody designed.
What the Technology Leader Owns
Everything above can be started by an engineering manager. Three things cannot, because they are decisions about how the organization judges its own investment.
Choose the number the board hears
Somebody will report on AI's return. If the technology leader does not choose the measure, the easiest one wins: code volume or tool adoption. Pick the delivery measures before the next board pack, show the baseline, report the trend. A flat line honestly reported beats a rising line that measures typing.
Decide where the saved time goes
Left alone, time saved writing code goes into writing more code, which feeds the queue. A deliberate leader moves some of it to the constraint: review capacity, test environments, clearer specifications. The study found firms already drifting this way, with the share of workers doing reviews up 14%. The question is whether the shift is designed or accidental. The same saved time also removes many of the everyday reasons teams used to talk to each other, which I wrote about in Silo Engineering Is Back.
Do not cut on keyboard metrics
The study found no significant change in employment attributable to AI, and ruled out a fall in overall employment of more than about 3% among agent adopters. If you size teams on the assumption that 30% more code means 30% fewer engineers, you are making a structural decision on a number that never reached production. Decide on staffing after you see the delivery trend, not before.
Architecture governance has the same shape. In a Series A SaaS company I worked with, decisions were slow not because people were slow, but because everything went through the same heavy gate. Once the decision rights were explicit, decision cycles moved from two to four weeks down to two to five days. AI-assisted delivery has the same shape. The gain is real at one stage, and it only reaches the business when someone redesigns the stage after it.
What the Market Is Showing
Three signals point the same way.
The evidence
The Chen and Stratton paper, Artificial Intelligence in the Firm: Bottlenecks in Software Production, is a working paper, not a peer-reviewed result, and its firms all use an engineering analytics platform, which is not every company. Its headline is not that AI fails. It is that coding productivity rises and the gain does not pass through, because review becomes the bottleneck.
The conversation
The Hacker News thread on the study stood at 364 points and 313 comments at the time of writing. It is a developer-heavy audience, so read it as a practitioner signal. The disagreement in it is useful: one camp wants review to shrink into specs and tests, the other sees review as the place where a team keeps understanding its own system. A leader does not have to pick a camp. They have to decide which changes need which kind of review.
The job market
Employers are already writing this into senior roles. In a base of 26 senior architecture, AI and engineering leadership postings reviewed earlier this month, 6 asked for an engineering operating model with measurement built in. That is my own tagging, so treat it as the shape of demand rather than a statistic. Two I re-opened today make the point plainly:
- A CTO role at Channel Fusion asks for a weekly pod-level scorecard, a controlled AI-assisted delivery pilot launched within the first 30 days, and measurable improvement in cycle time or throughput.
- A VP Engineering role at LRQA asks the hire to define the engineering performance and productivity framework, adopt AI-assisted engineering across teams on sound foundations, and show outcomes through DORA and cycle-time metrics.
Neither asks for more code. Both ask for someone who can prove AI moved delivery with the guardrails intact.
Action Plan
This Week
Ask three questions about the teams that use AI coding tools today:
- When you report on AI's impact, is the number about code produced or about work reaching production?
- Over the last month, has the time pull requests spend waiting for review gone up or down, and does anyone know?
- What share of changes is being sent back for rework, and is it higher for AI-assisted work than for the rest?
If nobody can answer the second question without building a report first, you do not yet know where your constraint is.
Next 30 Days
Trace twenty recent changes from first commit to production and record where each one waited. Set a baseline for the five scorecard measures: work finished and deployed, lead time, review wait and review time, rework share, and change failure rate. Agree on the risk tiers for review on one team, and write them down where reviewers will see them.
3 to 6 Months
Report the delivery scorecard to leadership on a fixed rhythm alongside, or instead of, tool adoption numbers. Move the rules that never change into automated checks. Shift capacity deliberately toward the constraint, whether that is review, test environments or specification. Revisit staffing assumptions only once the delivery trend is visible across at least a quarter.
Final Thought
The plant with the faster machine is not failing. The machine works. What it produces is waiting in front of an inspection station that nobody resized, and the dashboard on the machine is the one being shown to the board.
AI coding tools are that machine. They do make writing code faster. What they cannot do is redesign the pipeline around them, choose the measure that matters, or decide where the saved time goes. Those are leadership decisions.
So the useful question is not how much more code your teams produce. It is how much more software reaches the people it was built for, and which stage is stopping the rest.
If you can answer that, AI is a real gain waiting to be released. If you cannot, it is a bigger pile in front of the same door.
Measure What Ships Before You Report What AI Returned
If your AI coding rollout is reported in lines, commits or acceptance rates and nobody can say whether more work is reaching production, that is not an adoption problem. It is a measurement and constraint problem, and it is cheapest to fix before the next budget or staffing decision is made on the wrong number.
or
More code is an input. Shipped software is the output. Measure the output.
