Automate Your Code Reviews With AI: CodiumAI vs Github Copilot

Have you ever found yourself struggling to remember the modifications you made to a forked project? Maybe the changes were extensive, and despite your efforts, you worry that you might have missed capturing all the details in the pull request comment. Or perhaps you've been reviewing changes in a codebase and felt lost because the contributors didn't provide sufficient context for their modifications. In this post, I will demonstrate an AI tool that can help you review PRs more quickly and effectively.

In the world of software development, Code reviews are not just a checkbox or an unpleasant routine to add more work, they are important to maintain a quality, reliable, and consistent codebase especially when working as a team. Imagine you just built an app to power the next big thing, picture this without a code review, it's like an artist sending a masterpiece into the hands of users without a final polish, a spellcheck, or a friendly critique. It might work, but chances are, there could be a hidden bug, or even a potential failure waiting to happen.

Pull Request

A pull request is used in version control systems such as Git to propose changes to a codebase. A Developer working on a local branch of a codebase, implementing new features, fixing bugs, or making improvements, will need to make a pull request to merge those changes into the main or target branch, including a comment to provide context.
This pull request now becomes the focal point for code reviews. The good news is that there are AI tools that can assist you in reviewing PRs more quickly and efficiently.

CodiumAI PR_Agent Vs GitHub Copilot

We've seen the use of AI in virtually everything in the tech space, with the rise of GPT, a lot of solutions to everyday problems are been made.CodiumAI is one of the interesting solutions that caught a lot of developers' attention and I will be comparing its features with Github Copilot helping you choose the right tool for your next pull request.

What is CodiumAI's PR-Agent?

CodiumAI has developed a powerful tool, known as PR-Agent, designed to assist developers in speeding up and enhancing the efficiency of their pull request (PR) reviews. This AI agent automatically scrutinizes the commits and the PR itself, offering various types of valuable feedback:

Auto-Description: Automatically generates a PR description, including details such as the name, type, summary, and a walkthrough of the code.

PR Review: Provides feedback on the main theme of the PR, its type, relevant tests, potential security issues, and other suggestions to improve the PR content.

Question Answering: Capable of answering free-text questions related to the PR, offering insights and information.

Code Suggestion: Offers committable code suggestions aimed at enhancing the overall integrity of the PR, from bug identification to providing meaningful tests.

The interesting thing about this tool is that it's free, it's open-sourced, it supports six command prompts and it supports all git environments!

Github Copilot for Pull Requests

GitHub Copilot's pull request summaries, an AI-powered feature, empower users to craft a comprehensive overview of changes in a pull request. This includes detailing the impacted files and guiding reviewers on key focus areas during the review process.

Upon a user's request for a summary, Copilot meticulously scans the pull request. It presents an overview of the changes in a narrative format, supplemented by a concise, bulleted list specifying the changes and their respective impacted files.

Leverage GitHub Copilot seamlessly to generate pull request summaries directly on GitHub.com. This functionality serves a dual purpose—assisting reviewers in understanding modifications and providing a quick grasp for users conducting a pull request review.

GitHub Copilot's analysis of the pull request ensures a detailed overview in narrative form and a succinct, bulleted list outlining changes and their impacted files. This summary feature is conveniently accessible in various locations, offering flexibility in its utilization.

Feature-by-Feature Comparison

Real-world Use Cases

Currently, Copilot Pull request summaries is in beta and is only available to a limited number of users, however, I would what I found on their website.

CodiumAI's PR-Agent

Picture this: we're a dynamic team on a mission to fortify our website with an impenetrable password validator. We're tasked with crafting a program that takes a string as input and decides if it's a rock-solid password. Now, what makes a password truly formidable? Well, we've set some ground rules: it should boast a minimum of 2 numbers, 2 special characters from the elite squad ('!', '@', '#', '$', '%', '&', '*'), and a length of at least 7 characters. If a password passes this litmus test, it earns the prestigious title of 'Strong'; otherwise, it's dubbed 'Weak.'

Let see our eager junior dev, who dives straight into the code fray with this valiant attempt:

string = input()
num_count = 0
char_count = 0
char  = ["!","@","#","$","%","&","*"]
nums  = ["1","2","3","4","5","6","7","8","9","0"]

for var in string:
    if var in char:
        char_count  += 1
    if var in nums:
        num_count  += 1


if (char_count>1 and num_count>1) and (len(string)>6):
    print("Strong")

else:
    print("Weak")

Bravo, right? Well, not quite. There's a little hiccup. The code's loyalty to the "in" operator is a bit too exclusive—it checks only one value in a sequence and refuses to mingle with "and" to check multiple values.

But fear not! Another team member, ever watchful for such nuances, stepped in, wrote a test code to unveil the error, and gracefully submitted a pull request. let's see PR-Agent in play:

One good aspect of the PR-Agent tool is its user-friendly nature. To initiate any command, simply tag @CodiumAI-Agent and choose from the available commands: /describe, /review, /improve, or /ask. Let's kick things off with a review:

This is incredible! CodiumAI's PR_Agent provided a thorough review, giving the team a clear understanding of the situation.

Imagine the possibilities if we could engage PR-Agent to assist us with addressing the issues. This versatile tool not only helps in crafting detailed summaries but can also contribute to code improvements. All it takes is a simple comment to tap into its capabilities, Let's ask for help with improving the code:

"@CodiumAI-Agent /improve --pr_code_suggestions.extra_instructions="make the code a function and correct the validator's logic"

I've just tagged the PR-Agent tool, issued a command to enhance the code, and included specific instructions to turn it into a function and rectify the logic. I'm simply delegating all the work to PR_Agent. Check this out—it's pretty cool!

Not only have I saved time by avoiding the need to explain my actions in a comment, but now I also have a tool that can assist me in refining my code on the fly.

Let's ask PR_Agent to help us with naming the file, considering that well-chosen filenames can greatly enhance context:

GitHub Copilot

Currently, the GitHub Copilot Pull request summary is in beta and is only available to a limited number of users, however, I have read through their docs and I will share the capabilities available with the tool.
Sure, let's break it down:

1. Start on GitHub:

   • Go to GitHub.com, where you can manage and collaborate on your code.

2. Create or Open a Pull Request:

   • If you're making a new set of changes, create a pull request. If there's already one in progress, find and open it.

3. Blank Description is Best:

   • Make sure the description of your pull request is blank. GitHub Copilot works best when starting with a clean slate.

4. Find the Summary Field:

   • Locate the place where you want to add a summary. This could be in the description when creating a new pull request, in the opening comment of an existing one, or as a separate comment at the bottom of the pull request page.

5. Adding a Summary:

   • In the field where you're typing, look for options at the top. Find the one that looks like a bullet point, and click on it. This is where you'll find the option to add a summary.

.
It's important for users participating in a beta to be aware that the feature may still have some rough edges, and unexpected issues may arise. Beta testing is a collaborative effort between developers and users to refine and polish a feature before it reaches the broader user community in a stable and fully released version.

Which tool is best for me?

The choice of the right tool depends on various factors mentioned in the post. Considerations such as pricing preference, whether for a paid or free tool, play a role. While some may lean towards open-source tools, it's essential to evaluate specific features. Open-source tools, like the ones I prefer, offer accessibility to the source code, fostering transparency and community collaboration. This approach allows for code review, audits, and suggested improvements, expediting the identification and resolution of security vulnerabilities.

Another aspect to ponder is the range of supported commands in both tools. If you only need a bulleted list of changes, GitHub Copilot could be a suitable choice. However, if you require additional features like assistance in improving your work or seeking guidance through an 'AI Assistant' for specific code aspects, PR-Agent may be more suitable.

PR-Agent provides developers and repository maintainers with valuable information, streamlining the PR approval process. It also offers actionable code suggestions to enhance the quality of the PR. This integrated approach allows developers to assess the impact of their work without leaving their preferred Git provider environment, whether it's GitHub, GitLab, or another platform.

Conclusion

Deciding among these tools depends on what you need, what your project aims to achieve, and what you like. Whether opting for the collaborative power of open source or the concise capabilities of a dedicated code summarization tool, there are now different options to make coding even better. The important thing is to adapt to new tools that fit your project and team well. Whether you lean towards the detailed insights of PR-Agent or the succinct summaries of GitHub Copilot, the overarching goal remains the same: to help developers and make reviewing code together easier and more effective.