I have taken up an rather challenging task. That is take some ugly enterprise C# code and translate it to Python. Before I do a deep dive, I feel I need to do some explaining.
This C# project serves as an data plumbing layer for some machine learning models. See you hear data the first thing that should come in your mind is Python. Unfortunately this was not the case, and to make it better it is using the .NET extension for Apache Spark. Migrating it to Python makes sense, since it is using rather non-standard technology for the problem, which makes hiring future talent rather difficult. And there are other obvious problems, common in other enterprise OOP languages as well, like mulitple levels of abstraction, inheritance everywhere, inversion of control, again things you do not do in data plumbing. And to futher feed the devil, the original team of contractors/consultants (because who else would use C#, than some enterprice consultant) are going to leave next month.
BTW, i use Neovim, and I have been using Github Copilot for nearly two years, also Copilot Chat (technically ChatGPT) since I got my hands on the preview version, so I am by no means an total newbie, when it comes to LLMs and code.
The meat!
What does it feel like to code with Claude 3.5, well there are some good, bad, and ugly parts. But mostly you have no idea what is happening, even If you are able to write the same functionality from scratch.
The Good
Claude 3.5 tend produces suprisingly correct results, however, sometimes it tries to be smart, and it fails, but you can guide it to fix it issues. Overall it is a good companion to ask questions, and for me, since I had an strickt workflow and I did put a lot of effort to craft an sofisticated prompt with a lot of examples and edgecases I was more than satisfied. I could take an C# class and translate the important parts into Python that required very little additional instructions.
The bad
I usually can go and write code for 12 hours, with ocasional bathroom breaks and to refill my watter canteen. And yes, I do not feel hunger when I really get into it. With Claude I did get into the flow, but after something like 3 hours I just felt dead. During these 3 hours I did produce a lot of code, and I felt productive, but it was like doing nonstop code reviews, asking for changes, and most ofthen or not looking up the documentation if what was generated is even valid.
Ugly
If you are an coder, you are probably to some degree familiar to code you wrote. This familiary tends to fade with time, but if you work sufficiently long with a codebase you know your way arround. With LLM generated code this just does not happen. I had very little understanding what was going on. For days I did produce a lot of code, reviewed every line, from time to time I requested some change, but In the end I had little to no idea what is actually going on. At the end the codebase felt like a stranger, something I did not write. And to make things worse, the actuall code style was inconsistent, you could tell appart what files where written on the same day, It felt like different people wrote it.
Takeway
I spent a lot of time to review, rething, tweak this full blown LLM coding approach, however the result most of the time ist just code, that somewhat works, but you have no idea why or how. The big question is:
Is it worth it to end up with a code base you have little attachment to, that immediately feels like you wrote it a year ago with one time imaginary collegues.
Just to sum it up, I did feel I produced more code, however the quality was lower than usual. In some cases this lower quality is justifiable, especially if you do not care. But what I fear, that the time is coming, where every codebase will become "legacy" in record time.
Top comments (0)