DEV Community

Cover image for CoduimAI's New Cover-Agent: The Ultimate Guide
Nnaemeka Daniel John
Nnaemeka Daniel John

Posted on

CoduimAI's New Cover-Agent: The Ultimate Guide

The Cover-Agent test coverage tool just dropped!!

Big mood

So, I'll be walking you through everything you need to know in order to get Cover-Agent up and running locally on your workstation.

Overview of Cover-Agent

First, let's understand what the Cover-Agent is and what it does.

The Cover-Agent is an opensource test coverage tool by CodiumAI that utilizes various Generative AI models to automate unit test creation for your projects.
Under the hood, the Cover-Agent is comprised of various components, namely;

  • The Test Runner: Which runs the test suite and generates code coverage reports.
  • The Coverage Parser: Which verifies that code coverage increases as tests are added, ensuring that new tests contribute to the overall test effectiveness.
  • The Prompt Builder: Which gathers the necessary data from the codebase and constructs the prompt to be passed to the Large Language Model.
  • The Caller: Which interacts with the LLM to generate tests based on the prompt provided.

Cover-Agent can currently be run locally on your terminal, although CodiumAI is working on making it compatible with popular CI platforms.


Before you begin, ensure you have the following:

  • OPENAI_API_KEY: Required for calling the OpenAI API.
  • Python/PIP: Required for installing Cover-Agent.

To get Cover-Agent running as fast as possible, the easiest setup would be to install Cover-Agent as a Python Pip package using the command.

$ pip install git+
Enter fullscreen mode Exit fullscreen mode

To ensure that Cover-Agent have been properly installed, you can run the cover-agent command on the terminal, and you should see something like this.

$ cover-agent
usage: cover-agent [-h] --source-file-path SOURCE_FILE_PATH --test-file-path TEST_FILE_PATH --code-coverage-report-path CODE_COVERAGE_REPORT_PATH --test-command TEST_COMMAND
                   [--test-command-dir TEST_COMMAND_DIR] [--included-files [INCLUDED_FILES ...]] [--coverage-type COVERAGE_TYPE] [--report-filepath REPORT_FILEPATH]
                   [--desired-coverage DESIRED_COVERAGE] [--max-iterations MAX_ITERATIONS] [--additional-instructions ADDITIONAL_INSTRUCTIONS] [--openai-model OPENAI_MODEL] [--prompt-only]
cover-agent: error: the following arguments are required: --source-file-path, --test-file-path, --code-coverage-report-path, --test-command
Enter fullscreen mode Exit fullscreen mode

Next, you need to set your OpenAI key in your environment variables.

$ export OPENAI_API_KEY=**********************
Enter fullscreen mode Exit fullscreen mode

Generating Tests using Cover-Agent

Now you are ready to run the Cover Agent to generate and validate unit tests.
You can execute it from the command line using the following command:

$ cover-agent \
  --source-file-path "<path_to_source_file>" \
  --test-file-path "<path_to_test_file>" \
  --code-coverage-report-path "<path_to_coverage_report>" \
  --test-command "<test_command_to_run>" \
  --test-command-dir "<directory_to_run_test_command>" \
  --coverage-type "<type_of_coverage_report>" \
  --desired-coverage <desired_coverage_between_0_and_100> \
  --max-iterations <max_number_of_llm_iterations> \
  --included-files "<optional_list_of_files_to_include>"
Enter fullscreen mode Exit fullscreen mode

Running Cover-Agent

I have a Python project which auto-generates portfolios using the GitHub API. The API endpoints in this project are located in my file.

I also have a test file, for testing the API endpoints.

Test file

So with this setup I’m ready to run cover-agent from the terminal.


I ran the command below to generate relevant unit tests for my project.

$ cover-agent \
  --source-file-path "" \
  --test-file-path "" \
  --code-coverage-report-path "coverage.xml" \
  --test-command "pytest --cov=. --cov-report=xml --cov-report=term" \
  --test-command-dir "." \
  --coverage-type "cobertura" \
  --desired-coverage 70 \
  --max-iterations 10
Enter fullscreen mode Exit fullscreen mode

Ensure you replace the file paths with the actual file path where your code and test files are located.


Cover-Agent appends the generated unit tests to your test file.

Test file

After generating tests for your project, you’ll notice some new files in your repo.

  • Which contains the full prompt that is sent to the LLM.


  • coverage.xml: Which contains the test coverage scope.


  • run.log: Which contains a copy of the logger that gets dumped to your stdout.

Run log

  • test_results.html: Which contains the results table that contains the following for each generated test:
  • Test status
  • Failure reason (if applicable)
  • Exit code,
  • stderr
  • stdout
  • Generated test


Testing is no doubt one of the most important aspects of software development. And with a few steps you can up your testing game by utilizing the Cover-Agent tool to generate meaningful tests for your projects, ensuring test coverage for your codebase in no time.

Cover-Agent is an opensource software on GitHub, which means you can take a look at the inner workings of the tool, make meaningful contributions and help extend the capabilities of the tool.


Top comments (0)