What is the real effect of GitHub Copilot on developer efficiency?
Recently, a study analysed 2,631 survey responses from developers using GitHub Copilot, based on the SPACE framework, comparing their feedback with data from the integrated development environment (IDE). The study found that the rate of accepting suggested changes serves as a more precise measure of perceived efficiency.
The SPACE framework outlines five aspects of productivity: Satisfaction and well-being, Performance, Activity, Communication and collaboration, and Efficiency and flow. Excluding Activity, the study explored these domains through 11 statements. Participants were asked questions regarding their proficiency with the programming language most used with Copilot and their overall programming experience.
Key findings include:
- GitHub Copilot significantly boosts developer efficiency across all levels of experience, with the most notable benefits seen in junior developers.
- The benefits of AI-assisted coding have been confirmed across various productivity aspects, such as task length, output quality, cognitive load, satisfaction, and learning.
- While expert engineers may not necessarily write better code with Copilot, it can enhance their overall productivity.
- The value of Copilot's suggestions lies more in their ability to inspire further work than in their sheer accuracy.
- Junior developers report not only greater productivity improvements but also a higher rate of accepting suggestions.
- Observations show higher acceptance rates of Copilot's suggestions during weekends and post-work hours, after 4:00 pm.
Research paper here
Top comments (0)