A comparison of multiple generative AI tools when asking for Ruby on Rails code

Comparing Rix, Cody, Phind, Copilot Chat, and ChatGPT's responses to a basic Ruby on Rails query.

I asked the following query for multiple AI code generator tools:

Please transform this code:

<div class="flex justify-center py-6 mt-8 mb-3" role="separator">
  <span class="dot"></span>
  <span class="dot"></span>
  <span class="dot"></span>
</div>

into a Rails Helper with content tags

I tried to make the request a mix of simple wording while also providing a small context by specifying the content tags and mentioning Rails Helper.

Chat GPT

Here is the ChatGPT response:

It works, it will display the HTML as required.

My main review here is that I would use concat instead of join.html_safe

So I did a follow-up:

Of course, this was not what I was looking for, but I also agree that my request was not very specific. I did this intentionally to keep the conversation casual, and not suggest a specific solution. I also notice that it changed the second param of content_tag from an empty string to nil. I did not request this specific change but I assume asking refactor triggered a wider change.

I again follow up with the request to remove safe_join:

And this indeed is the solution that I think is a bit close to what I wanted. I don't think .map is needed there and I usually like helpers to make things explicit so I would have written 3 lines of concat(content_tag) but I think in general the solution is usable.

Phind

Phind describes themselves as "The AI search engine for developers".

Here is the response from Phind:

I was quite happy with the result:

• Good name dot_separator

• I like the idea that it extracted the actual dot to a private method dot_span but I think the name could have been improved to only dot.

• I like that it used concat

• And I welcome the explicitness of having 3 lines of contact(dot_span) instead of 3.times { contact(dot_span) }

I did not feel here that I need to follow up so I moved to the next tool.

Cody

Cody is an AI assistant from Source Graph - "Cody is an AI coding assistant that writes code and answers questions for you by reading your entire codebase and the code graph"

Here is the first response from Cody:

This will not work as intended, as it only displays one single dot.

So I said to Cody:

Now this code will not display anything, so I asked Cody to fix it:

Again this code will not show anything so I asked Cody again to fix it, but now I asked also to explain why it does not work. This is a trick with LLMs that will usually give better results because it brings more context about what is not working and the cause of it:

Now this works - it will display 3 dots, but it is still a version that I don't like as it uses html_safe. So I asked for a refactor:

This works and it is a solution that I consider ok(ish).

Github Copilot Chat

Github Copilot Chat is "A ChatGPT-like experience in your editor with GitHub Copilot chat" that I installed in a VScode Insiders version.

Here is the response I got from it:

First, this solution works. Then I like that it tried to give a good name, separator_dots, and I appreciate that it used concat instead of first trying to use html_safe

Copilot Chat is a version of the Open AI GPT-4 model, I think, but probably has a better system prompt.

Rix

Rix is a Hashnode tool described as "The AI search companion, optimized for developers"

Here is the response I got:

(temporary) conclusion

In the context of Ruby programming, AI code generator tools offer varying results and sometimes require follow-ups to achieve the desired output. While some tools, like Phind and Github Copilot Chat, provide satisfactory solutions quicker, others may need additional guidance. It is essential to thoroughly review the generated code to ensure security and avoid unintended changes.

Word of caution

Do not feed private information: I used this with a piece of code that does not show any private information about the project I worked with.

Be careful what code you feed into these tools: When using generative AI tools - I only add those to my side projects and most of the time when I know that I will anyhow make the project open source. I would not use them or give them access to a client code base unless explicitly accepted by the client in a contract.

Be careful of security issues: Also please be careful of copying/pasting code generated by AI in your project and running that code on servers. Nobody can guarantee that the code does not have security issues.

Read all the generated code, not just the changed parts: One last point, if you are asking for refactoring, read every time the entire code and not only the lines that you required to be changed. Sometimes the AI will change other things you did not specifically ask for.

Enjoyed this article?

Join my Short Ruby News newsletter for weekly Ruby updates. Also, check out my co-authored book, LintingRuby, for insights on automated code checks. For more Ruby learning resources, visit rubyandrails.info.