According to the Stack Overflow survey published in October 2026, 73% of the developers who responded use AI coding assistants and agents every day. Whether they actually work faster with them is a harder question. Controlled studies, in which developers are randomly assigned to work with or without AI, have reached very different results. We looked at what exactly each of them measured.
Source: Stack Overflow Developer Survey 2026
What did the individual studies measure?
| Study | Who and how many | What was measured | Result |
|---|---|---|---|
| GitHub and Microsoft Research, 2022 | 95 freelance programmers | time to complete one identical task | 55.8% faster with AI |
| Microsoft, Accenture and a Fortune 100 company | 4,867 developers in their day-to-day work | number of completed tasks | 26% more tasks with AI |
| Google, 2024 | 96 in-house engineers | time per task | roughly 21% faster, with a wide spread |
| METR, spring 2025 | 16 experienced developers, 246 real tasks | time per task in their own projects | 19% slower with AI |
The differences are not a mistake. Each study measured something different, with different people and different tools.
Sources: Peng et al., The Impact of AI on Developer Productivity: Evidence from GitHub Copilot, Cui et al., The Effects of Generative AI on High-Skilled Work, Paradis et al., How much does AI impact development speed?, METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Why did the first study come out so well?
The 2022 GitHub study gave everyone the same well-defined task: write a simple web server in JavaScript. Participants with the AI assistant finished in 71 minutes on average, those without it in 161 minutes. The authors themselves point out that this was a single standardised task, not work on a large project. The results also suggest that less experienced programmers got more out of AI.
Source: Peng et al., The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
What did the large companies find?
The largest trial took place at Microsoft, Accenture and another large company. Almost five thousand developers were randomly given an AI assistant or not, and the researchers followed their normal work. With AI they completed 26% more tasks on average. The differences between people were large: according to a summary by MIT Sloan, juniors and recent hires gained 27 to 39%, experienced developers only 8 to 13%. The study did not measure code quality, because the researchers had no access to the code.
Sources: Cui et al., The Effects of Generative AI on High-Skilled Work, MIT Sloan, How generative AI affects highly skilled workers
Why were experienced developers slower with AI?
The study that drew the most attention came from the non-profit METR in spring 2025. Sixteen experienced developers worked on real tasks in large open-source projects they had been contributing to for years. The projects averaged more than a million lines of code. With AI, the work took them 19% longer.
What stands out more is this: beforehand, the developers expected AI to speed them up by 24%, and even after finishing they believed it had made them 20% faster.
METR itself stresses that the result does not mean AI fails to speed up most developers, and it now considers the result out of date. The tools have changed a great deal since spring 2025.
Sources: METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR, We are Changing our Developer Productivity Experiment Design
Why is the experiment hard to repeat?
From August 2025, METR tried to repeat the experiment with 57 developers and ran into an unexpected problem. More and more developers declined to take part because they did not want to work without AI, and 30 to 50% of participants said in a survey that they had kept some tasks out of the study so they would not have to do them without AI. The new estimates favoured AI, but they are not statistically significant and METR described them as unreliable. According to METR, the speed-up could be much higher for the developers and tasks that dropped out of the study.
Source: METR, We are Changing our Developer Productivity Experiment Design
What happens to quality and skills?
Two more recent studies show the other side:
- Learning. In an Anthropic study from January 2026, 52 mostly junior engineers learned a new Python library. The group with AI scored 50% on average in a comprehension test, the group without AI 67%. The biggest gap was in questions on debugging. Those who used AI to understand the material did markedly better than those who only used it to generate code. The authors point out that this is a correlation in small groups, not a proven cause.
- Code complexity. In an observational study (not a randomised trial), researchers at Carnegie Mellon University compared 806 open-source projects that started using the Cursor editor with 1,380 projects that did not. In the first month, 281% more lines of code were added, but the speed-up had disappeared by the third month. Static analysis warnings rose by 30% and stayed higher.
Sources: Anthropic, How AI assistance impacts the formation of coding skills, He et al., Speed at the Cost of Quality
What does this mean for companies?
- Do not expect a single number. The benefit of AI depends on the type of work, people's experience and how well they know the project. It tends to be higher for new, well-defined tasks than in large systems that have been running for years.
- Measure rather than estimate. How fast people feel they are can be far from what a measurement shows. If you are introducing AI, track actual delivery time and quality, not only how satisfied people are.
- Keep an eye on quality and learning. Code written quickly can create more work later. Juniors need to use AI in a way that helps them learn, not as a substitute for learning.
Agents are taking on more and more of the work. According to GitHub's Octoverse report from October 2026, they opened more than 40% of pull requests on GitHub in September 2026. The report does not measure productivity or quality.
Source: GitHub Octoverse 2026
Summary
- Controlled studies measured gains of 21 to 56%, but also a 19% slowdown for experienced developers working in large projects.
- Developers often judge their gain from AI differently from what a measurement shows.
- Less experienced developers gain the most from AI. At the same time, risks are showing up for code quality and for learning.
If you are looking for developers who work with AI and keep code quality in check, write to us and tell us about the project. Ways of working with us are described on the contracting and AI solutions pages. How Czech companies use AI is covered in the article How many Czech companies use artificial intelligence.
As of 31 October 2026. Study results apply to the tools and conditions at the time each study was carried out.
- artificial intelligence
- software development
- productivity
- AI assistants
Sources
- Peng et al.: The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (arXiv, 2023)
- Cui et al.: The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers (Microsoft Research, 2025)
- MIT Sloan: How generative AI affects highly skilled workers (4 November 2024)
- Paradis et al.: How much does AI impact development speed? An enterprise-based randomized controlled trial (arXiv, 2024)
- METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (10 July 2025)
- METR: We are Changing our Developer Productivity Experiment Design (24 February 2026)
- Anthropic: How AI assistance impacts the formation of coding skills (29 January 2026)
- He et al.: Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects (arXiv, 2026)
- Stack Overflow Developer Survey 2026 (6 October 2026)
- GitHub Octoverse 2026 (28 October 2026)
