Malhar Ujawane

Inspiration

Inspiration·

Harvard: coding agents add 23% more PRs and 49% longer reviews

A Harvard study of 718 firms: coding agents raised lines of code 30% and PRs 23%, but shipped features didn't rise significantly as review time grew 49%.

Artificial Intelligence in the Firm: Bottlenecks in Software Production by Fiona Chen and James Stratton, Harvard University (current version August 4, 2026).

Chen and Stratton used Jellyfish engineering-analytics data covering about 300 million work events across 718 firms, from January 2021 to March 2026. They compared firms before and after they adopted AI coding assistants and AI coding agents. For agents, lines of code went up 30%, commits 20% and pull requests 23%.

Then they looked at what shipped. Resolved Jira issues and epics showed small, statistically insignificant increases, and for agents the estimates rule out an output increase larger than 12% of the baseline. Their explanation is the part I’d show any engineering manager:

We find evidence of a bottleneck from the code review process: The average time to review a pull request increases by 49%, the share of pull requests with changes requested nearly doubles, and the number of comments per pull request increases by 35%.

Firms responded by moving people into review: the share of workers doing code reviews went up 14%. AI review tools haven’t absorbed it yet. Nearly 80% of firms had used AI in code review by March 2026, but AI wrote only 23.3% of review comments, and only 10.8% of pull requests got at least one AI comment.

This matches what I’d expect from running delivery. An agent makes the writing step cheap, and every other step in the pipeline (scoping, review, QA, deploy) keeps its old cost. More pull requests at a lower average quality means more review, and review is done by the same senior people who were already the constraint.

If a client asks how much faster agents will make their team, I’d answer that it depends on review capacity, and this paper gives me numbers to point at. The changes I’d make follow from it: smaller agent PRs, a test or check that has to pass before a human looks at anything, and review time counted in the estimate. I wrote about giving each agent its own worktree and scoring every attempt with the same test, which is one way to filter before review. Anthropic’s study on analysts adopting their agent’s leanings is the other half of the problem: the reviewer’s judgment is the thing you’re paying for, so protect it.

The data ends in March 2026, so it predates the latest coding models. The authors also note the adoption measure is firm-level, which averages over people who barely use the tools. I’d still rather plan around this than around a vendor’s productivity chart.

Via Ars Technica, and the study was trending on X on October 28.

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.