DEV Community

Andrei Dascalu
Andrei Dascalu

Posted on

A little about Github Copilot

I've played with Github's copilot for a few days so I guess it's mandatory to write a few lines about.

I'll start with some general observations and then I'll add a few concerns I have with the approach.

Observations

  • Copilot isn't perfect. It's trained on public code and as such it's only as good as the public code is.
  • It's not copy/paste of public code but a sort of average take on very specific solutions.
  • Quality depends a A LOT on the language. I've tried it with Javascript, Go and PHP and I could (and I might) write a piece on Copilot with each of them. Javascript is pretty good (but it's not my main language). Go is great - underlining a bit that the language goes a long way to prevent you from shooting yourself in the foot and the majority of codebases tend to be idiomatic (this is reflected in what Copilot suggests). PHP, on the other hand, suffers from a lot of publicly available legacy codebases with old coding styles and the fact that Copilot doesn't quite infer from context the version of PHP that you're using. It may not be bad, depending on what you do with the suggestion received.
  • It can be annoying as while it trained on public code, it doesn't seem to pick up important things from local context. For example, I wrote a class and when I wrote a function making use of some private member, the suggestion made use of it nicely. The implementation (though simple) was perfect. However, a moment later I wanted to wrap it in a try/catch. Instead of wrapping the existing code, Copilot suggested a whole new block of code: duplicating the existing solution, but inside a try catch (eg: try { } catch () {} ). Rather annoying when trying to fix up existing code.

Concerns

  • I can see how Copilot can be very very bad for beginners. Particularly as underlined for PHP, Copilot is as good as the codebases it learned from. With PHP, Copilot produces a lot of ancient-looking code despite surrounding code using PHP 8 features. It's VERY tempting to simply accept the provided solutions, because they do work.
  • I can see how Copilot can be good for beginners. To me, it makes sense to use Copilot in the same way you're using Stackoverflow code: as an example to build on. However, this presents another challenge: resist the urge to use code as-is and focus on adjusting the code according to your standards (aka: the project's). It's difficult because the code works and makes sense as-is, but can still be messy.
  • Copilot can be bad in the sense that it puts you (a developer in need of help) into a box. It's box formed by the code Copilot has learned from. A regular internet search can provide you with a variety of ideas to choose from, whereas Copilot chooses for you. It's expediting the process but depriving you of a source of learning: simply confronting and weighing different solutions helps your evolution as a developer.
  • You can get different solutions from Copilot but at the moment there's no straightforward way to compare them directly.

Cheers!

Oldest comments (0)