CodiumAI PR-Agent: The Copilot Killer?

In the rapidly evolving landscape of AI-driven coding assistance, two tools stand out: CodiumAI PR-Agent and GitHub Copilot. Both offer unique features and benefits, and in this comparison, we'll explore the strengths, weaknesses, and unique features of these two AI-powered tools.

Let's see who emerges victorious in this ultimate showdown!

CodiumAI PR-Agent Overview

CodiumAI's PR-Agent is an open-source tool that leverages AI technology to assist developers in reviewing pull requests more efficiently and effectively. It offers several commands such as /describe, /review, /ask, and /improve to analyze code changes, identify potential issues, and provide insightful feedback.

One of the key strengths of CodiumAI PR-Agent is its extensive command support, which allows for a wide range of code analysis and improvement tasks. Its open-source nature also allows developers to contribute to its development and customize it according to their specific needs. Furthermore, CodiumAI PR-Agent supports multiple git providers (GitHub, Gitlab, Bitbucket), multiple ways to use the tool (CLI, GitHub Action, GitHub App, Docker, etc.), and multiple models (GPT-4, GPT-3.5, Anthropic, Cohere, Llama2)

GitHub Copilot for Pull-Request Overview

On the other hand, GitHub Copilot is a tool that uses AI to provide suggestions for code as developers type. GitHub Copilot primarily relies on autocomplete suggestions during coding sessions. It offers real-time coding assistance and completion, focusing on suggesting and completing code snippets while coding.

While it doesn't offer the same level of extensibility as CodiumAI PR-Agent, it does provide useful features for PRs. For instance, it can generate PR descriptions, and provide feedback on the main theme, type, relevant tests, security issues, score, and various suggestions for the PR content.

However, unlike CodiumAI PR-Agent, GitHub Copilot operates on a single command, which limits its functionality and flexibility. Additionally, as a closed-code in-house project, GitHub Copilot lacks the open-source availability of CodiumAI PR-Agent, which restricts its flexibility and customization.

Comparison

While both tools offer valuable features, they differ in several key aspects:

• Functionality: GitHub Copilot is primarily focused on code completion and suggestion, whereas CodiumAI PR-Agent is designed specifically for handling pull requests.
• Community Involvement: As an open-source tool, CodiumAI PR-Agent encourages community involvement, allowing developers to contribute to its development, report issues, and enhance its capabilities. GitHub Copilot, on the other hand, follows a proprietary approach, which may limit direct contributions to its codebase.
• Customization and Flexibility: CodiumAI PR-Agent, due to its open-source nature, provides developers with greater flexibility and customization options. GitHub Copilot, while focusing on providing concise, AI-generated descriptions, may not offer the same level of customization.
• Pricing: Codium PR-Agent is free to use, while GitHub Copilot costs $10 USD per month or $100 USD per year. There is also a free tier for students and educators.

Hands-on Experience

I have used GitHub copilot for Code Completion and Code Generation and it was very impressive but I could not use co-pilot for pull requests.

Let's see how CodiumAI performs on pull requests.

For this experiment, I integrated CodiumAI agent in my small project in github using this installation guide to run codium as a github action. It took me less than 5 mins to set it up.

I created a very small change in my python project (Jarvis) and added 2 basic util methods.

Then I created a pull request: https://github.com/piyushchhabra/jarvis/pull/2

I intentionally kept meaningless summary and description on my PR

As soon as the PR was created, I observed Codium agent action running. Within a minute, my pull request was analysed and following details were added on the PR:

It pointed out that I have not added any tests for my change.

Hmm.. Interesting 🤔

Now let's see if it can actually generate a meaningful summary and description on my PR. I added a comment on PR describe

Another GitHub action ran, and I see that my pull request summary as well as description was updated to below

That's pretty great for this change 👏🏻

Conclusion

While both CodiumAI PR-Agent and GitHub Copilot have their unique strengths, CodiumAI PR-Agent appears to offer more flexibility and functionality due to its extensive command support and open-source nature. However, the choice between the two would ultimately depend on the specific needs and preferences of the developer or team.

I would personally place my bets on CodiumAI, primarily because it being open-sourced and more flexible in terms of developer experience.

What do you guys think..? Let me know in the comment section.