+994512060920
In recent years, AI-powered coding tools have significantly transformed the workflows of software engineers. Developers now rely on AI tools to automatically write code, fix errors, and test changes. This has had a major impact on the software industry. Many articles — and developers themselves — claim that these tools enhance speed and productivity.
But how accurate are these claims?
A recent study by METR — a non-profit AI research organization — offers surprising insights into this question. In the study, 16 experienced open-source developers were asked to complete 246 real-world tasks taken from large codebases they regularly contribute to. The tasks were randomly divided into two groups: in one group, developers were allowed to use advanced AI tools like Cursor Pro (“AI-allowed”), and in the other group, the use of AI tools was prohibited.
Before the study began, the developers were asked to estimate how much faster they would be with AI. On average, they predicted that AI assistance would reduce task completion time by 24%.
The results, however, were unexpected.
As the data shows, when developers were allowed to use AI, task completion time actually increased by 19% — meaning they worked more slowly with AI tools.
This study casts doubt on the widely assumed productivity gains promised by AI. Several factors may have contributed to this result:
Developers spent more time prompting the AI and waiting for responses than actually writing code.
AI tools struggled with large and complex codebases.
The researchers behind the study caution against drawing overly broad conclusions. They note that AI is rapidly evolving, and these results may no longer hold true even a few months from now. In other research, METR has found that AI coding tools have recently improved in handling complex, long-term tasks.
In conclusion, while AI shows great promise, its productivity benefits should be approached with caution. Developers should not assume that AI will instantly speed up their workflows, and they should use these tools thoughtfully and critically.
