Update
At the time of the article, there was no way to retrieve the combined status for commit checks and check runs. But now there is
The final, updated code would no look like this
const QUERY = `query($owner: String!, $repo: String!, $pull_number: Int!) {
repository(owner: $owner, name:$repo) {
pullRequest(number:$pull_number) {
commits(last: 1) {
nodes {
commit {
statusCheckRollup {
state
}
}
}
}
}
}
}`
async function getCombinedSuccess(octokit, { owner, repo, pull_number}) {
const result = await octokit.graphql(query, { owner, repo, pull_number });
const [{ commit: lastCommit }] = result.repository.pullRequest.commits.nodes;
return lastCommit.statusCheckRollup.state === "SUCCESS"
}
In this post, you will learn
- Where the pull request checks are coming from
- There is no single API endpoint to retrieve the combined status for a pull requests
- The difference between Commit Status, Check Runs, and GitHub Action results
- How to get a combined status for a pull request
Storytime
I'm a big fan of automation. In order to keep all dependencies of my projects up-to-date, I use a GitHub App called Greenkeeper. It creates pull requests if there is a new version of a dependency that is out of range of what I defined in my package.json files.
This is a huge help, I could not maintain as many Open Source libraries if it was not for Greenkeeper and other automation.
However, whenever there is a new breaking version of a library that I depend on in most of my projects, I get 100s of notifications for pull requests, all of which I have to review and merge manually. After doing that a few times, I decided to create a script that can merge all pull requests from Greenkeeper that I got notifications for. I'd only need to check it once to make sure the new version is legit, all other pull requests should just be merged, as long as the pull request is green (meaning, all tests & other integrations report back with a success status).
Turns out, "as long as the pull request is green" is easier said than done.
What is a pull request status?
The first thing that is important to understand is where the list of checks shown at the bottom of most pull requests on GitHub is coming from.
Pull request checks are not set on pull requests. They are set on the last commit belonging to a pull request.
If you push another commit, all the checks will disappear from that list. The integrations that set them will need to set them again for the new commit. This is important to understand if you try to retrieve the checks using GitHub's REST or GraphQL APIs. First, you need the pull request's last commit (the "head commit"), then you can get the checks.
What is the difference between commit statuses and check runs
Commit statuses was the original way for integrators to report back a status on a commit. They were introduced in 2012. Creating a commit status is simple. Here is a code example using @octokit/request
import { request } from '@octokit/request'
// https://developer.github.com/v3/repos/statuses/#create-a-status
request('POST /repos/:owner/:repo/statuses/:commit_sha', {
headers: {
authorization: `token ${TOKEN}`
},
owner: 'octocat',
repo: 'hello-world',
commit_sha: 'abcd123',
state: 'success',
description: 'All tests passed',
target_url: 'https://my-ci.com/octocat/hello-world/build/123'
})
And retrieving the combined status for a commit is as just as simple
import { request } from '@octokit/request'
// https://developer.github.com/v3/repos/statuses/#get-the-combined-status-for-a-specific-ref
request('GET /repos/:owner/:repo/commits/:commit_sha/status', {
headers: {
authorization: `token ${TOKEN}`
},
owner: 'octocat',
repo: 'hello-world',
commit_sha: 'abcd123'
})
.then(response => console.log(response.data.state))
But with the introduction of check runs in 2018, a new way was introduced to add a status to a commit, entirely separated from commit statuses. Instead of setting a target_url, check runs have a UI integrated in github.com. Integrators can set an extensive description. In many cases, they don't need to create a separate website and exclusively use the check runs UI instead.
Creating a check run is a bit more involved
import { request } from '@octokit/request'
// https://developer.github.com/v3/checks/runs/#create-a-check-run
request('POST /repos/:owner/:repo/check-runs', {
headers: {
authorization: `token ${TOKEN}`
},
owner: 'octocat',
repo: 'hello-world',
name: 'My CI',
head_sha: 'abcd123', // this is the commit sha
status: 'completed',
conclusion: 'success',
output: {
title: 'All tests passed',
summary: '123 out of 123 tests passed in 1:23 minutes',
// more options: https://developer.github.com/v3/checks/runs/#output-object
}
})
Unfortunately, there is no way to retrieve a combined status from all check runs, you will have to retrieve them all and go through one by one. Note that the List check runs for a specific ref endpoint does paginate, so I'd recommend using the Octokit paginate plugin
import { Octokit } from '@octokit/core'
import { paginate } from '@octokit/plugin-paginate-rest'
const MyOctokit = Octokit.plugin(paginate)
const octokit = new MyOctokit({ auth: TOKEN})
// https://developer.github.com/v3/checks/runs/#list-check-runs-for-a-specific-ref
octokit.paginate('GET /repos/:owner/:repo/commits/:ref/check-runs', (response) => response.data.conclusion)
.then(conclusions => {
const success = conclusions.every(conclusion => conclusion === success)
})
A status reported by a GitHub Action is also a check run, so you will retrieve status from actions the same way.
How to retrieve the combined status for a pull request
You will have to retrieve both, the combined status of commit statuses and the combined status of check runs. Given you know the repository and the pull request number, the code would look like this using @octokit/core with the paginate plugin
async function getCombinedSuccess(octokit, { owner, repo, pull_number}) {
// https://developer.github.com/v3/pulls/#get-a-single-pull-request
const { data: { head: { sha: commit_sha } } } = await octokit.request('GET /repos/:owner/:repo/pulls/:pull_number', {
owner,
repo,
pull_number
})
// https://developer.github.com/v3/repos/statuses/#get-the-combined-status-for-a-specific-ref
const { data: { state: commitStatusState } } = request('GET /repos/:owner/:repo/commits/:commit_sha/status', {
owner,
repo,
commit_sha
})
// https://developer.github.com/v3/checks/runs/#list-check-runs-for-a-specific-ref
const conclusions = await octokit.paginate('GET /repos/:owner/:repo/commits/:ref/check-runs', {
owner,
repo,
commit_sha
}, (response) => response.data.conclusion)
const allChecksSuccess = conclusions => conclusions.every(conclusion => conclusion === success)
return commitStatusState === 'success' && allChecksSuccess
}
Using GraphQL, you will only have to send one request. But keep in mind that octokit.graphql does not come with a solution for pagination, because it's complicated™. If you expect more than 100 check runs, you'll have to use the REST API or look into paginating the results from GraphQL (I recommend watching Rea Loretta's fantastic talk on Advanced patterns for GitHub's GraphQL API to learn how to do that, and why it's so complicated).
const QUERY = query($owner: String!, $repo: String!, $pull_number: Int!) {
repository(owner: $owner, name:$repo) {
pullRequest(number:$pull_number) {
commits(last: 1) {
nodes {
commit {
checkSuites(first: 100) {
nodes {
checkRuns(first: 100) {
nodes {
name
conclusion
permalink
}
}
}
}
status {
state
contexts {
state
targetUrl
description
context
}
}
}
}
}
}
}
}
async function getCombinedSuccess(octokit, { owner, repo, pull_number}) {
const result = await octokit.graphql(query, { owner, repo, pull_number });
const [{ commit: lastCommit }] = result.repository.pullRequest.commits.nodes;
const allChecksSuccess = [].concat(
...lastCommit.checkSuites.nodes.map(node => node.checkRuns.nodes)
).every(checkRun => checkRun.conclusion === "SUCCESS")
const allStatusesSuccess = lastCommit.status.contexts.every(status => status.state === "SUCCESS");
return allStatusesSuccess || allChecksSuccess
}
See it in action
I use the GraphQL version in my script to merge all open pull requests by Greenkeeper that I have unread notifications for: merge-greenkeeper-prs.
Happy automated pull request status checking & merging 🥳
Credit
The header image is by WOCinTech Chat, licensed under CC BY-SA 2.0
Top comments (4)
Very nice presentation thx. Anyway I have a problem using github that maybe you'd have some idea to explore. I use Pull Request and a workflow on github, the Pull Request has check-runs defined. When I run this workflow manually (using workflow_dispatch) the check-run statuses are not updated. I found an article that explain why (github.community/t/workflow-dispat...), but I don't find any way (nor idea) on how to make a workaround for that ...
Maybe you would have some idea ?
Many thx.
Great, helpful article. I'm wondering if the (new?) StatusCheckRollup object changes any of this?
Thanks Rhys, I've updated the article!
Fantastic post - thank you