GitHub API: How to retrieve the combined pull request status from commit statuses, check runs, and GitHub Action results

#github #javascript #octokit

Update

At the time of the article, there was no way to retrieve the combined status for commit checks and check runs. But now there is

The final, updated code would no look like this

const QUERY = `query($owner: String!, $repo: String!, $pull_number: Int!) {
  repository(owner: $owner, name:$repo) {
    pullRequest(number:$pull_number) {
      commits(last: 1) {
        nodes {
          commit {
            statusCheckRollup {
              state
            }
          }
        }
      }
    }
  }
}`

async function getCombinedSuccess(octokit, { owner, repo, pull_number}) {
  const result = await octokit.graphql(query, { owner, repo, pull_number });
  const [{ commit: lastCommit }] = result.repository.pullRequest.commits.nodes;
  return lastCommit.statusCheckRollup.state === "SUCCESS"
}

In this post, you will learn

Where the pull request checks are coming from
There is no single API endpoint to retrieve the combined status for a pull requests
The difference between Commit Status, Check Runs, and GitHub Action results
How to get a combined status for a pull request

Storytime

I'm a big fan of automation. In order to keep all dependencies of my projects up-to-date, I use a GitHub App called Greenkeeper. It creates pull requests if there is a new version of a dependency that is out of range of what I defined in my package.json files.

This is a huge help, I could not maintain as many Open Source libraries if it was not for Greenkeeper and other automation.

However, whenever there is a new breaking version of a library that I depend on in most of my projects, I get 100s of notifications for pull requests, all of which I have to review and merge manually. After doing that a few times, I decided to create a script that can merge all pull requests from Greenkeeper that I got notifications for. I'd only need to check it once to make sure the new version is legit, all other pull requests should just be merged, as long as the pull request is green (meaning, all tests & other integrations report back with a success status).

Turns out, "as long as the pull request is green" is easier said than done.

What is a pull request status?

The first thing that is important to understand is where the list of checks shown at the bottom of most pull requests on GitHub is coming from.

Pull request checks are not set on pull requests. They are set on the last commit belonging to a pull request.

If you push another commit, all the checks will disappear from that list. The integrations that set them will need to set them again for the new commit. This is important to understand if you try to retrieve the checks using GitHub's REST or GraphQL APIs. First, you need the pull request's last commit (the "head commit"), then you can get the checks.

What is the difference between commit statuses and check runs

Commit statuses was the original way for integrators to report back a status on a commit. They were introduced in 2012. Creating a commit status is simple. Here is a code example using @octokit/request

import { request } from '@octokit/request'

// https://developer.github.com/v3/repos/statuses/#create-a-status
request('POST /repos/:owner/:repo/statuses/:commit_sha', {
  headers: {
    authorization: `token ${TOKEN}`
  },
  owner: 'octocat',
  repo: 'hello-world',
  commit_sha: 'abcd123',
  state: 'success',
  description: 'All tests passed',
  target_url: 'https://my-ci.com/octocat/hello-world/build/123'
})

And retrieving the combined status for a commit is as just as simple

import { request } from '@octokit/request'

// https://developer.github.com/v3/repos/statuses/#get-the-combined-status-for-a-specific-ref
request('GET /repos/:owner/:repo/commits/:commit_sha/status', {
  headers: {
    authorization: `token ${TOKEN}`
  },
  owner: 'octocat',
  repo: 'hello-world',
  commit_sha: 'abcd123'
})
  .then(response => console.log(response.data.state))

But with the introduction of check runs in 2018, a new way was introduced to add a status to a commit, entirely separated from commit statuses. Instead of setting a target_url, check runs have a UI integrated in github.com. Integrators can set an extensive description. In many cases, they don't need to create a separate website and exclusively use the check runs UI instead.

Creating a check run is a bit more involved

import { request } from '@octokit/request'

// https://developer.github.com/v3/checks/runs/#create-a-check-run
request('POST /repos/:owner/:repo/check-runs', {
  headers: {
    authorization: `token ${TOKEN}`
  },
  owner: 'octocat',
  repo: 'hello-world',
  name: 'My CI',
  head_sha: 'abcd123', // this is the commit sha
  status: 'completed',
  conclusion: 'success',
  output: {
    title: 'All tests passed',
    summary: '123 out of 123 tests passed in 1:23 minutes',
    // more options: https://developer.github.com/v3/checks/runs/#output-object
  }
})

Unfortunately, there is no way to retrieve a combined status from all check runs, you will have to retrieve them all and go through one by one. Note that the List check runs for a specific ref endpoint does paginate, so I'd recommend using the Octokit paginate plugin

import { Octokit } from '@octokit/core'
import { paginate } from '@octokit/plugin-paginate-rest'

const MyOctokit = Octokit.plugin(paginate)
const octokit = new MyOctokit({ auth: TOKEN})

// https://developer.github.com/v3/checks/runs/#list-check-runs-for-a-specific-ref
octokit.paginate('GET /repos/:owner/:repo/commits/:ref/check-runs', (response) => response.data.conclusion)
  .then(conclusions => {
    const success = conclusions.every(conclusion => conclusion === success)
  })

A status reported by a GitHub Action is also a check run, so you will retrieve status from actions the same way.

How to retrieve the combined status for a pull request

You will have to retrieve both, the combined status of commit statuses and the combined status of check runs. Given you know the repository and the pull request number, the code would look like this using @octokit/core with the paginate plugin

async function getCombinedSuccess(octokit, { owner, repo, pull_number}) {
  // https://developer.github.com/v3/pulls/#get-a-single-pull-request
  const { data: { head: { sha: commit_sha } } } = await octokit.request('GET /repos/:owner/:repo/pulls/:pull_number', {
    owner,
    repo,
    pull_number
  })

  // https://developer.github.com/v3/repos/statuses/#get-the-combined-status-for-a-specific-ref
  const { data: { state: commitStatusState } } = request('GET /repos/:owner/:repo/commits/:commit_sha/status', {
    owner,
    repo,
    commit_sha
  })

  // https://developer.github.com/v3/checks/runs/#list-check-runs-for-a-specific-ref
  const conclusions = await octokit.paginate('GET /repos/:owner/:repo/commits/:ref/check-runs', {
    owner,
    repo,
    commit_sha
  }, (response) => response.data.conclusion)

  const allChecksSuccess = conclusions => conclusions.every(conclusion => conclusion === success)

  return commitStatusState === 'success' && allChecksSuccess
}

Using GraphQL, you will only have to send one request. But keep in mind that octokit.graphql does not come with a solution for pagination, because it's complicated™. If you expect more than 100 check runs, you'll have to use the REST API or look into paginating the results from GraphQL (I recommend watching Rea Loretta's fantastic talk on Advanced patterns for GitHub's GraphQL API to learn how to do that, and why it's so complicated).

const QUERY = `query($owner: String!, $repo: String!, $pull_number: Int!) {
  repository(owner: $owner, name:$repo) {
    pullRequest(number:$pull_number) {
      commits(last: 1) {
        nodes {
          commit {
            checkSuites(first: 100) {
              nodes {
                checkRuns(first: 100) {
                  nodes {
                    name
                    conclusion
                    permalink
                  }
                }
              }
            }
            status {
              state
              contexts {
                state
                targetUrl
                description
                context
              }
            }
          }
        }
      }
    }
  }
}`

async function getCombinedSuccess(octokit, { owner, repo, pull_number}) {
  const result = await octokit.graphql(query, { owner, repo, pull_number });
  const [{ commit: lastCommit }] = result.repository.pullRequest.commits.nodes;

  const allChecksSuccess = [].concat(
    ...lastCommit.checkSuites.nodes.map(node => node.checkRuns.nodes)
  ).every(checkRun => checkRun.conclusion === "SUCCESS")
  const allStatusesSuccess = lastCommit.status.contexts.every(status => status.state === "SUCCESS");

  return allStatusesSuccess || allChecksSuccess
}

See it in action

I use the GraphQL version in my script to merge all open pull requests by Greenkeeper that I have unread notifications for: merge-greenkeeper-prs.

Happy automated pull request status checking & merging 🥳

Credit

The header image is by WOCinTech Chat, licensed under CC BY-SA 2.0

Top comments (4)

maximequinzin • Apr 27 '22

Very nice presentation thx. Anyway I have a problem using github that maybe you'd have some idea to explore. I use Pull Request and a workflow on github, the Pull Request has check-runs defined. When I run this workflow manually (using workflow_dispatch) the check-run statuses are not updated. I found an article that explain why (github.community/t/workflow-dispat...), but I don't find any way (nor idea) on how to make a workaround for that ...
Maybe you would have some idea ?
Many thx.