Generating a feature is only the beginning. Somebody still has to understand it, verify its behavior and decide whether it belongs in a release.
That distinction is central to a new edition of our software-productivity coverage: more coding activity and more finished value are different outcomes.
What the larger study found
Fiona Chen and James Stratton’s August 4, 2026 working paper analyzes roughly 300 million workplace events across 718 firms, from January 2021 through March 2026. Using staggered adoption comparisons, it estimates these changes after AI-agent adoption: [1]
More output. A longer path through review.
Rounded relative changes; different measures, not a quality ranking. Point estimates shown without confidence intervals. Source: Chen & Stratton, August 2026, pp. 3 and 22.
Review duration is elapsed time between submission and merge, including waiting, not measured active reviewer hours. Output here means resolved Jira issues and epics. A statistically insignificant increase does not prove zero benefit. [1]
The observational design relies on comparability assumptions; it is not randomized. Its data cutoff also excludes subsequent tool improvements. Treat the result as evidence about these firms and this period, not a permanent verdict on AI.
The counterevidence matters
A separate peer-reviewed study in Management Science, published February 27, 2026, pooled randomized experiments involving 4,867 developers at Microsoft, Accenture and another Fortune 100 firm. It reported a 26.08% increase in completed tasks with a code-completion assistant, with substantial uncertainty and variation across experiments. [2]
That is not a head-to-head test against the agent study. Individual assistance and firm adoption are different treatments. Completed tasks and company-wide production are different measurements.
Known, uncertain and speculation
KNOWN
The evidence differs.
Reported results depend on the tool, setting and outcome measured. Neither study establishes the productivity of every team.
UNCERTAIN
What happens next?
How newer agents affect production quality and long-term maintenance remains unresolved by these results.
SPECULATION
The bottleneck can move.
Better automated review might turn more generated code into useful delivery. That is a hypothesis to test.
What to measure on your own team
Our practical recommendation: evaluate the complete delivery path. The following checklist is editorial guidance, not an experiment we conducted.
| Measure | What it helps you understand |
|---|---|
| Lead time | How long review-ready work takes to reach production. |
| Review and rework | The human effort and corrections required for approval. |
| Release outcomes | Whether the intended user behavior works after deployment. |
| Escaped defects | Which failures reached users and what repairs cost. |
Compare similar changes before and after adoption, record their size and risk, and count time spent correcting output. A small experiment in one workflow can be more informative for your team than a broad claim that AI makes every engineer faster.
For developers, explain the intent and assumptions behind a change. For QA, derive expected behavior from requirements and check integrations. For leaders, give review and testing capacity the same attention as authoring speed.
A useful AI workflow should help a team ship something reliable that people actually need.
Primary sources
- Chen & Stratton — Artificial Intelligence in the Firm: Bottlenecks in Software Production · August 4, 2026 version · Working paper. Data supplied by Jellyfish.
- Cui et al. — The Effects of Generative AI on High-Skilled Work · February 27, 2026 · Management Science; randomized field experiments.