DEV Community

epassaro
epassaro

Posted on • Edited on

Deploy your package documentation on GitHub Pages in the RTD style: preview PRs, and more!

#actionshackathon21 #github

Overview

In the recent years, lot of Python packages moved their documentation pipelines from third party services like ReadTheDocs to GitHub Pages, but some useful features were missing in the process.

In this post, I'm going to describe how to write an -almost- complete RTD pipeline for GitHub Actions.

Features

  1. Support of pip and conda dependency files.
  2. Build and deploy branches under /branches/<branch>.
  3. Build and deploy tags under /tags/<tag>
  4. Secure build and preview of pull requests under /pull/<number> via labels.
  5. Manual trigger for branches.
  6. Select branch to deploy the site (default: gh-pages).
  7. Redirect the main domain to /latest or /stable (default: /latest).
  8. Redirect the /stable subdomain to the latest SemVer tag, or the stable branch (if exists).
  9. Automatic removal of pages from closed/merged pull requests, and deleted branches or tags.

Not supported

  1. Multi-language builds.
  2. ???

Live demo

The workflow

Triggers

The pipeline should be triggered on the following events:

  1. After pushing commits to main and other specified branches.
  2. After pushing a semantic versioned tag to the repository.
  3. After (re)opening, labeling or pushing commits on a PR using the pull_request_target event to allow sharing secrets. This behavior is constrained later at job level for security reasons.
  4. Manually, from the GitHub Actions tab. 
on:

  push:
    branches:
      - main                            # default branch
      - new-feature                     # extra branches to build
    tags:
      - 'v[0-9]+.[0-9]+.[0-9]+'         # semantic versions

  pull_request_target:                  # pull request build
    branches:
      - '*'
    types:
      - opened
      - synchronize
      - reopened
      - labeled                         # requires the `build-docs` label

  workflow_dispatch:                    # manual trigger
    branches:
      - '*'
Enter fullscreen mode Exit fullscreen mode

Parameters

These are the parameters you can tweak from the env section.

  • PYTHON: the Python version (default: 3.9)
  • PKGS_FILE: a txt or yml file, depending if you want to use conda or pip to deploy your environment.
  • BUILD_CMD: the Sphinx build command (default: cd docs && make html).
  • DEPLOY_BRANCH: your GitHub Pages branch (default: gh-pages)
  • ROOT_REDIRECT: redirect the root URL to /latest or /stable (default: /latest, the build of default branch). 
env:
  PYTHON: 3.9
  PKGS_FILE: docs/requirements.txt      # .txt (`pip`) or .yml (`conda`) file
  BUILD_CMD: cd docs/ && make html
  DEPLOY_BRANCH: gh-pages               #  target branch to deploy _build/html
  ROOT_REDIRECT: latest                   # `latest` or `stable`
Enter fullscreen mode Exit fullscreen mode

The build job

Sharing secrets safely

The if condition at the top of the build job constraints the pull_request_target trigger by requiring the build-docs label trigger the job.

Only people with write access to a repository can label pull requests, so this a nice way to allow sharing secrets safely with trustworthy contributors.

jobs:
  build:

    if: github.event_name == 'push' ||
        github.event_name == 'workflow_dispatch' ||
        (github.event_name == 'pull_request_target' &&
        contains(github.event.pull_request.labels.*.name, 'build-docs'))

    runs-on: ubuntu-latest
    steps:

      - uses: actions/checkout@v2
        if: github.event_name != 'pull_request_target'

      - name: Checkout pull request ${{ github.event.number }}
        uses: actions/checkout@v2
        with:
          ref: ${{ github.event.pull_request.head.sha }}
        if: github.event_name == 'pull_request_target'
Enter fullscreen mode Exit fullscreen mode

Note that malicious changes to the code (in this case the Sphinx Makefile) can lead to all kinds of security issues, so make sure to review the PRs of untrusted contributors before labeling them.

For more information, see: "Keeping your GitHub Actions and workflows secure"

Check existing branches

This checks for the existence of the deploy and stable branches to be used later.

      - name: Check branches
        run: |
          CHECK_DEPLOY=$(git ls-remote --heads origin refs/heads/${{ env.DEPLOY_BRANCH }})
          CHECK_STABLE=$(git ls-remote --heads origin refs/heads/stable)
          echo "::set-output name=DEPLOY::$(! [[ -z $CHECK_DEPLOY ]] && echo 'true' || echo 'false')"
          echo "::set-output name=STABLE::$(! [[ -z $CHECK_STABLE ]] && echo 'true' || echo 'false')"          
          cat $GITHUB_ENV
        id: check-branches
Enter fullscreen mode Exit fullscreen mode

Environment setup

In the next step we setup Mambaforge, a custom Miniconda installer that ships the blazing-fast mamba package manager.

Then, the environment is updated depending on the extension of the provided file.

      - name: Setup Mambaforge
        uses: conda-incubator/setup-miniconda@v2
        with:
            miniforge-variant: Mambaforge
            miniforge-version: latest
            activate-environment: sphinx
            python-version: ${{ env.PYTHON }}
            use-mamba: true

      - name: Install packages
        run: |
          if [[ ${{ env.PKGS_FILE }} == *.txt ]]; then
            pip install -r ${{ env.PKGS_FILE }}

          elif [[ ${{ env.PKGS_FILE }} == *.yml ]] || [[ ${{ env.PKGS_FILE }} == *.yaml ]]; then
            mamba env update -n sphinx -f ${{ env.PKGS_FILE }}

          else
            echo "Unsupported file extension"
            exit 1

          fi
Enter fullscreen mode Exit fullscreen mode

Build and deploy

Since the deployment of the site depends on the combinations of the event triggers, I needed to write a not-so-complicated bash script to handle the destination path appropriately.

Then, the peaceiris/actions-gh-pages action deploys the site on the resulting $DEST_DIR variable.

      - name: Build documentation
        run: ${{ env.BUILD_CMD }}

      - name: Set destination directory
        run: |          
          BRANCH=$(echo ${GITHUB_REF#refs/heads/})
          TAG=$(echo ${GITHUB_REF#refs/tags/})

          if [[ $EVENT == push ]] || [[ $EVENT == workflow_dispatch ]]; then
            if [[ -z $TAG ]] || [[ $TAG == $GITHUB_REF ]]; then

              if [[ $BRANCH == $DEFAULT ]]; then
                echo "DEST_DIR=latest" >> $GITHUB_ENV
              else
                echo "DEST_DIR=branch/$BRANCH" >> $GITHUB_ENV
              fi

            elif [[ ! -z $TAG ]]; then
               echo "DEST_DIR=tag/$TAG" >> $GITHUB_ENV

            else
              echo "Unexpected ref $GITHUB_REF"
              exit 1
            fi

          elif [[ $EVENT == pull_request_target ]]; then
            echo "DEST_DIR=pull/$PR" >> $GITHUB_ENV

          else
            echo "Unexpected event trigger $EVENT"
            exit 1

          fi
          cat $GITHUB_ENV
        env:
          DEFAULT: ${{ github.event.repository.default_branch }}
          EVENT: ${{ github.event_name }}
          PR: ${{ github.event.number }}

      - name: Deploy ${{ env.DEST_DIR }}
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_branch: ${{ env.DEPLOY_BRANCH }}
          publish_dir: ./docs/_build/html
          destination_dir: ${{ env.DEST_DIR }}          
          user_name: 'github-actions[bot]'
          user_email: 'github-actions[bot]@users.noreply.github.com'
Enter fullscreen mode Exit fullscreen mode

Redirects

Finally, we need to redirect the main URL and the /stable subdomain according to the parameters selected above.

      - name: Redirect root
        run: |
          mkdir redirects && cd redirects
          echo '<head>' >> index.html
          echo '  <meta http-equiv="Refresh" content="0; url='/${{ github.event.repository.name }}/${{ env.ROOT_REDIRECT }}'"/>' >> index.html
          echo '</head>' >> index.html

      - name: Redirect stable
        run: |
          git fetch origin
          git checkout ${{ env.DEPLOY_BRANCH }}

          if [[ -d tag ]] && [[ ! -z $(ls -A tag) ]]; then
            LAST_TAG=$(ls -d tag/*/ | sort -V | tail -n 1)
            mv $LAST_TAG /tmp/stable

            git checkout -
            mv /tmp/stable redirects
            ls -lR redirects

          else
            echo "No tags to deploy"

          fi
        if: steps.check-branches.outputs.DEPLOY == 'true' && steps.check-branches.outputs.STABLE == 'false'

      - name: Deploy redirects
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./redirects
          keep_files: true
          user_name: 'github-actions[bot]'
          user_email: 'github-actions[bot]@users.noreply.github.com'
Enter fullscreen mode Exit fullscreen mode

The clean workflow

A separate workflow handles the removal of pages from merged/closed pull requests, deleted branches or tags.

#  If you use this workflow, please acknowledge it
#
#  author:          Ezequiel Pássaro (@epassaro)
#  organization:    TARDIS-SN (@tardis-sn)
#  license:         MIT

name: clean-docs

on:

  delete:
    branches:                           # remove deleted branches or tags
      - '*'
    tag:
      - '*'

  pull_request_target:                  # remove closed or merged pull requests
    branches:
      - '*'
    types:
      - closed

env:
  DEPLOY_BRANCH: gh-pages               #  deployed docs branch

jobs:
  clean:
    runs-on: ubuntu-latest
    steps:

      - uses: actions/checkout@v2

      - name: Set folder to delete
        run: |
          if [[ $EVENT == delete ]]; then
            echo "DEST_DIR=$EVENT_TYPE/$EVENT_REF" >> $GITHUB_ENV

          elif [[ $EVENT == pull_request_target ]]; then
            echo "DEST_DIR=pull/$PR" >> $GITHUB_ENV

          else
            echo "Unexpected event trigger $EVENT"
            exit 1

          fi
          cat $GITHUB_ENV
        env:
          EVENT: ${{ github.event_name }}
          EVENT_REF: ${{ github.event.ref }}
          EVENT_TYPE: ${{ github.event.ref_type }}
          PR: ${{ github.event.number }}

      - name: Clean ${{ env.DEST_DIR }}
        run: |
          git fetch origin ${{ env.DEPLOY_BRANCH }}
          git checkout ${{ env.DEPLOY_BRANCH }}
          git config user.name github-actions[bot]
          git config user.email github-actions[bot]@users.noreply.github.com

          if [[ -d $DEST_DIR ]]; then
            git rm -rf $DEST_DIR
            git commit -m "clean $DEST_DIR"
            git push

          else
            echo "$DEST_DIR does not exist"

          fi
Enter fullscreen mode Exit fullscreen mode

Get the code

GitHub logo epassaro / actions-rtd-workflow

A RTD-like documentation pipeline for GitHub Actions

actions-rtd-workflow

A RTD-like documentation pipeline for GitHub Actions

Features

  1. Support of pip and conda dependency files.
  2. Build and deploy branches under /branches/<branch>.
  3. Build and deploy tags under /tags/<tag>.
  4. Secure build and preview of pull requests under /pull/<number> via labels.
  5. Manual trigger for branches.
  6. Select branch to deploy the site (default: gh-pages).
  7. Redirect the main domain to /latest or /stable (default: /latest).
  8. Redirect the /stable subdomain to the latest SemVer tag, or the stable branch (if exists).
  9. Automatic removal of pages from closed/merged pull requests, and deleted branches or tags.

Not supported

Multi-language builds.

Live demo




View on GitHub


Top comments (0)

Read next

kodebae profile image

How to remove a leaked .env file from GitHub permanently...

Kodebae -

travislramos profile image

The Essential Git Cheat Sheet: Master Version Control Like a Pro

Travis Ramos -

jonasbn profile image

Hosting Swagger-UI using GitHub Pages

Jonas Brømsø -

7u5h4r profile image

How react works in backend

Tushar Sahani -