How automating PR review infra helped Hasura with shipping docs

The need for automation

In all fields, repetitive tasks abound. However, in the software industry, we're fortunate (or cursed) to have the knowledge and tools to automate away different aspects of our work. Hasurians were recently recommended the book Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations (Forsgren et al., 2018). At the heart of this book is the message that what can be automated should be automated.

Accelerate focuses on several key points as to why we should automate tasks:

• Humans get bored: computers succeed at doing simple tasks quickly. On the other hand, humans are problem solvers. If we can offload repetitive tasks to a piece of software, we are free to focus on more engaging and exciting work.
• Humans are prone to error: especially when we're bored! With a concrete set of conditional logic, given the right APIs, most simple tasks can be handled by a piece of software.
• Humans can only sustain a pace for so long: critical to high-performing organizations is the ability to regularly deploy with as little friction and fatigue as possible. Computers should take as much of this load as possible to help humans maintain cadence.

It's no surprise that these points overlap and feed into each other. An engineer, bored with work, is bound to make mistakes. Those mistakes trip up a release cadence and cause the organization to perform sub-optimally. This pattern repeats itself, and - before long - burnout overcomes this engineer.

As Hasura is OSS at heart, the Documentation (Docs) team receives a lot of PRs from both the community and members of Hasura. With the other tasks we're working on during a sprint, it can be challenging to keep up with the incoming review requests. Add to this the firehose-level deluge of services competing for our attention and pinging us about new PRs (GitHub notifications, internal messages on Slack, Discord, Gmail), and we're bound to miss something.

How to automate your PR infrastructure

We utilize Jira to track tasks during a sprint and map out our upcoming work. When designing this automation, we knew we wanted to achieve several goals:

• Create a single source of truth: instead of parsing through notifications from different services, we wanted to be able to look at our board and know what PRs were up for review and dependent on us.
• Offload the responsibility: instead of tasking team members with periodically checking for new PRs, we wanted to free ourselves from the repetitive grind of creating an issue, linking to the PR, and adding it to the board.
• Deliver quicker reviews: instead of letting a PR sit in some limbo state, we wanted to provide feedback as quickly as possible to ensure updates merged swiftly.

To meet these goals, we designed a GitHub Action. This Action looks for PRs containing changes to files in the /docs directory of the Hasura GraphQL Engine and automatically creates a ticket in Jira for us.

What we'll build below is a model of that same Action. We'll utilize Jira and GitHub Actions to build an automated PR-ticketing system for easy capture and tracking. While it's helpful to be familiar with YAML, it's an easy language to pick up and is intended to be human-readable. Follow the steps below to start automating your PR review infra 🤙

Step 1: Generate a token and store secrets

To begin, you'll need a Jira API token. You'll use this token to authenticate your Action and link your Jira board with a GitHub repository. Generating a token is a straightforward process that Atlassian explains in their documentation; follow these steps.

Once you've followed the steps above, we need to store this and other pertinent information inside the repository's secrets. GitHub secrets are used similarly to environment variables. Instead of hardcoding these values in our Action, we'll reference them using a familiar syntax and ensure our API key is safe. The three pieces of information we'll store as secrets are:

• JIRA_API_TOKEN: The Jira API token you generated earlier.
• JIRA_BASE_URL: The Jira board's base URL.
• JIRA_USER_EMAIL: The email address associated with the user account generating the ticket (most likely yours or a service account belonging to your organization).

You can access your repository's secrets by going to the repository's settings. From here, in the Security section, choose Secrets and then Actions.

Step 2: Write the Action boilerplate

GitHub Actions all follow a common implementation pattern. Within any repository, you'll begin by creating a .github directory and - within this folder - a subdirectory called workflows. This workflows directory is where your Actions will live.

Design of Actions should be limited in scope to unique tasks or jobs you're seeking to automate. You may have numerous Actions listed as separate YAML files within this workflows directory.

Begin by naming the Action descriptively (as these names will appear whenever your CI runs). For our example, we'll create a new file called create-docs-ticket.yml and fill it with the following boilerplate to begin:

name: create docs ticket
on:
  pull_request:
    types: [opened]
    paths:
      - 'docs/**'

jobs:
  create-docs-ticket:
    runs-on: ubuntu-latest

The name key is a copy of the kebab-cased filename we used earlier. Actions allow us to utilize different events as triggers. For our purposes, we're concerned with new PRs opened on our repository; thus, we're using the [opened] value on the types key. Finally, the jobs listed are the set of tasks, in order, that need to run for our automation to occur.

To ensure we only generate tickets on updates to files in the /docs directory, we utilize the paths key and tell the Action we only want the following jobs to run if a PR makes changes to files - of any type - inside this directory.

Step 3: Bring in Jira templates

We don't have to write all of our Actions' logic from scratch. A marketplace exists where you can find numerous templates and Actions that take care of specific, common tasks. GitHub designed Actions to be modular, so we can quickly piece together prebuilt Actions that solve common problems and create unique solutions. We'll utilize two Atlassian-generated Actions to authenticate who we are and then make the Jira ticket:

jobs:
  create-docs-ticket:
    runs-on: ubuntu-latest
    steps:
      - name: Login
        uses: atlassian/gajira-login@master
        env:
          JIRA_BASE_URL: ${{ secrets.JIRA_BASE_URL }}
          JIRA_USER_EMAIL: ${{ secrets.JIRA_USER_EMAIL }}
          JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}

      - name: Create ticket
        id: create
        uses: atlassian/gajira-create@master

Using the above values will ensure we can authenticate to Jira and have the necessary permissions to generate a ticket on our board. From here, we must tell the Action to which board it should add the ticket and which values to populate inside it. Our board is named DOCS, so we use that as the value on project. You should update this to the name of your board / project in Jira. Actions can also handle dynamic values, so we can reference information from the PR using strings for the summary and description keys:

jobs:
  create-docs-ticket:
    runs-on: ubuntu-latest
    steps:
      - name: Login
        uses: atlassian/gajira-login@master
        env:
          JIRA_BASE_URL: ${{ secrets.JIRA_BASE_URL }}
          JIRA_USER_EMAIL: ${{ secrets.JIRA_USER_EMAIL }}
          JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}

      - name: Create ticket
        id: create
        uses: atlassian/gajira-create@master
        with:
          project: DOCS
          issuetype: Review
          summary: Review ${{ github.event.pull_request.title }}
          description: ${{ github.event.pull_request.html_url }}

In all, the Action file should look like this:

name: create-docs-ticket
on:
  pull_request:
    types: [opened]
    paths:
      - 'docs/**'

jobs:
  create-docs-ticket:
    runs-on: ubuntu-latest
    steps:
      - name: Login
        uses: atlassian/gajira-login@master
        env:
          JIRA_BASE_URL: ${{ secrets.JIRA_BASE_URL }}
          JIRA_USER_EMAIL: ${{ secrets.JIRA_USER_EMAIL }}
          JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}

      - name: Create ticket
        id: create
        uses: atlassian/gajira-create@master
        with:
          project: DOCS
          issuetype: Review
          summary: Review ${{ github.event.pull_request.title }}
          description: ${{ github.event.pull_request.html_url }}

      - name: Log created issue
        run: echo "Issue ${{ steps.create.outputs.issue }} was created for docs team to review your PR 🤙"

Step 4: Test it out

At this point, you're ready to go! Give it a run by creating a PR that modifies a file based on whatever condition you set. Whenever someone opens a PR, you should see the Action run as a part of your repository's CI. Head over to your Jira board and see if there's a new ticket waiting for you!

Community-facing results

As a team, we're happy with the results. Not only have we decreased our lower-order workload, but the first-order implications for those we work with and all of you have been positive. While we have a steady stream of backlogged issues and PRs to work through, we know this measure will improve our responsiveness to all new PRs opened since its introduction.

Internal results

One recent trend within the company has been how we collectively examine and provide feedback on pull requests. As a core value, ownership plays a vital role in ensuring we hold ourselves accountable and continue to ship improvements to the product. While everyone within our team is listed as a 'code owner' to the files within /docs, we don't operate with one collective brain. Simply assigning Docs to review changes to documentation leaves some room to be desired. To put it bluntly, "The fastest way to starve a horse is to assign two people to feed it."

With that in mind, we'll be taking our GitHub Action one step further. The Docs team has two engineers; we've split the load to be "on call" for new PRs. Using Jira's REST API, our Action will eventually determine who is on call whenever a new PR is opened and assign that engineer to be the reviewer.

Put it to the test!

We want you to create some PRs and put this Action to work! With Hacktoberfest starting soon, the different teams at Hasura have been hard at work creating good-first-issues. We in the Docs team could certainly use some help from all of you, especially first-time contributors! If you've never contributed to open-source, consider checking out our list of issues. We're here to help and appreciate your support!