AI coding agents generate more code, but not more software

研究发现，编码效率的提升被人肉审查的“瓶颈”所“吸收”。 Study finds coding efficiency gains get “absorbed” by human review “bottleneck.”

来，我帮你……嗯，要不要我搭把手？图片来源：Getty Images

Here, let me just... yeah, can I lend you a hand here? Credit: Getty Images

任何哪怕稍微接触过计算机编程的人都知道，现代 AI 编程助手和智能体在生成大量可运行代码方面效率惊人。但使用这些工具的程序员也很清楚不能盲信这些代码的准确性，这意味着需要花费大量精力去审查任何 AI 生成的输出。Anyone who has even tangentially associated with computer programming knows that modern AI coding assistants and agents can be incredibly efficient at generating huge amounts of functional code. But coders making use of those tools also know better than to trust the accuracy of that code, meaning substantial effort needs to be spent reviewing any AI-generated output.

一项针对数百家公司实际编程实践的近期研究发现，人工代码审查构成了 AI 编程工具整体效率的一大“瓶颈”，结果显示“几乎没有证据表明公司因使用这些工具而增加了软件产出或减少雇员”。研究作者发现，实际编码阶段提升的任何效率都会“被生产流程中的下游制约所吸收”；因为“代码审查流程的时长显著增加，拉取请求（pull request）更有可能被要求修改，审查者留下的评论也更多”。A recent study of actual coding practices across hundreds of firms finds that human code review forms a significant “bottleneck” for the overall efficiency of AI coding tools, resulting in “little evidence that firms increase software output or reduce employment” by using them. Any efficiency increased during the actual coding phase, the study authors find, is “absorbed by downstream constraints in the production process”; as “the code review process significantly increases in length, pull requests are more likely to require revisions, and reviewers leave more comments.”

先量两次，再剪一次Cut once, measure twice

为得出这些结论，哈佛大学研究人员 Fiona Chen 和 James Stratton 利用了 Jellyfish 的汇总分析数据，该公司可测量工程团队的细粒度产出。这些数据涵盖 2021 年至 2026 年 3 月间、来自 700 多家相关软件开发公司的 70 多万名员工的 3 亿条独立的“工作事件”（如提交和拉取请求），以及任务管理软件数据。To come to these conclusions, Harvard University researchers Fiona Chen and James Stratton made use of aggregated analytics data from Jellyfish, which measures the granular output of engineering teams. That data encompasses 300 million individual “work events” (e.g., commits and pull requests) and issue management software data across more than 700,000 employees at over 700 relevant software development firms from 2021 through March of 2026.

为评估 AI 工具对这些公司的影响，研究人员结合直接测得的 AI 使用数据和 GitHub 活动分析，确定每家公司何时开始在工作流中引入 AI 编程助手（主要帮助人类作者自动补全代码）和/或 AI 编程智能体（根据提示词自主编写并提交代码）。随后，研究人员通过一些复杂的数学计算，在不同组织于不同时间点引入这些工具前后的关键变量上进行“双重差分”回归分析。To assess the impact of AI tools on these firms, the researchers used a mix of directly measured AI usage and analyses of GitHub activity to determine when each company started introducing either AI coding assistants (which can help auto-complete code primarily authored by humans) and/or AI coding agents (which primarily write and submit code autonomously based on prompts) into their workflows. The researchers then perform some complicated math to determine a “difference of differences” regression on key variables both before and after the introduction of these tools at different points in time across different organizations.

If there’s one thing AI coding agents are good at, it’s churning out a lot of code. Credit: Chen and Stratton

就产出的原始代码而言，结果清晰而醒目。研究人员写道，公司引入 AI 编程智能体后，生成的代码总行数增加 30%，提交总数增加 20%，拉取请求平均增加 23%。但所有这些额外的代码并未直接转化为公司层面软件产出的提升。恰恰相反，Jira 等工具追踪的 Issue 和 Epic（即完整的软件功能）的解决率在引入 AI 工具后并未出现统计上显著的变化（研究人员还发现，在引入 AI 前后，这些 Jira 追踪的议题在规模或复杂度上也没有“结构性变化”）。In terms of raw code being produced, the results are clear and stark. The introduction of AI coding agents at a firm leads to a 30 percent increase in total lines of code generated, a 20 percent rise in the number of total commits, and a 23 percent increase in pull requests on average, the researchers write. But all that extra code doesn’t translate directly into improved software output on the firm level. On the contrary, the resolution rate for Issues and Epics (i.e. wholesale software features) tracked by tools like Jira did not change in a statistically significant way after AI tools were introduced (the researchers also found no “compositional shift” in the size or complexity of those Jira-tracked issues across the AI introduction).

这种落差的原因可以直接在代码审查环节找到——在引入 AI 编程智能体之后，审查耗时明显变长。总体来看，从拉取请求被提交到被合并进代码库的平均“审查流程”时间，在引入 AI 智能体后平均膨胀了 49%。研究人员写道，这一效应在更细粒度的数据中同样可见：AI 智能体变革之后，“被要求修改的拉取请求占比几乎翻倍，每个拉取请求的评论数增加了 35%”。The reason for that discrepancy can be found directly in the code review process, which takes markedly longer on average after the introduction of AI coding agents. Overall, the average “review process” time between a pull request getting submitted and it being merged into the codebase balloons 49 percent on average after AI agents are introduced. That effect can be seen in more granular data, too, with “the share of pull requests with changes requested nearly doubl[ing], and the number of comments per pull request increas[ing] by 35%” following the AI agent shift, the researchers write.

针对这一变化，研究人员发现引入 AI 智能体后，从事代码审查的员工占比增加了 14%。他们还写道，在统计 Jellyfish 上的活跃员工总数并与这些公司的 LinkedIn 数据交叉比对后，他们“无法将显著的雇员变化归因于 AI”。In response to this change, the researchers found a 14 percent increase in the share of workers performing code reviews after AI agents’ introduction. They also write that they “cannot attribute significant employment changes to AI” after looking at total active workers across Jellyfish and cross-referencing with LinkedIn data at those firms.

Pull requests need changes a lot more often in the “agentic coding” era. Credit: Chen and Stratton

尽管理论上 AI 也能辅助这一审查流程，但研究人员发现，到目前为止其影响微乎其微。尽管到 2026 年 3 月，受测公司中有 80% 使用了某种形式的 AI 代码审查，但 AI 智能体仅负责全部审查评论的 23.3% 和全部拉取请求的 10.8%，这表明绝大多数此类工作仍由人类承担。While AI could also theoretically help with this review process, the researchers found that, so far, that impact has been marginal. Although 80 percent of measured firms used some form of AI code review by March 2026, AI agents were only responsible for 23.3 percent of all review comments and 10.8 percent of all pull requests, suggesting humans were still responsible for the vast majority of this work.

当然，AI 智能体在编程领域还相对较新，即使在本研究 2026 年 3 月的数据截止之后，其输出质量也已有显著的更新和升级。而且虽然研究中 95% 的公司到这一时点都已部署 AI 编程智能体，但毫无疑问，许多公司仍在学习如何以及何时最佳地部署它们。随着软件工程团队对让 AI 智能体处理特定编码问题的利弊积累更多经验，这类“编码时间与审查时间”的权衡可能会有所改善。AI agents are still a relatively new part of the coding world, of course, and there have been significant updates and upgrades to their output even since this study’s March 2026 data cutoff. And while 95 percent of firms in the study have implemented AI coding agents by this point, many are doubtlessly still going through a learning process regarding when and how to best deploy them. These kinds of “coding time versus review time” trade-offs could improve as software engineering teams gain more experience with the pros and cons of siccing an AI agent on particular coding problems.

不过就目前而言，让 AI 编写代码看来是一把双刃剑：编码速度的提升被人工代码审查时间和精力几乎同等幅度的增加所抵消。这样的结果不禁让人怀疑，为了让 AI 编程智能体运转所投入的巨大时间和金钱，对大多数公司来说是否真的值得。For now, though, letting AI write your code looks like a double-edged sword, with increases in coding speed counteracted by similar increases in human code review time and effort. It’s the kind of result that makes us wonder if the considerable time and expense to get AI coding agents working is really worth it for most companies.