I work on a small team and we manage dozens of websites. With a host of requests coming in from Product Managers it's important that we work quickly without introducing new bugs into the code.

Of course, if you have a QA team you can rely on them to double-check issues, but we don't have that luxury. And even if you do, no QA engineer has ever complained about a layer of automated tests.

Enter Cypress Testing

If you're not familiar with Cypress, it's one of the newest and most popular testing frameworks to hit the scene recently. It's a lot like using selenium, with, so far, none of the annoyances. It even lets you record your own tests by clicking around your app and then later adding assertions! It literally writes the code!

You can run it locally while developing - which is great! - but I wanted to be able to ensure that before each PR is merged we could see test results to ensure it wouldn't bork our main development branch.

This guide will not walk you through how to setup Cypress, but check out the Getting Started pages.

You'll also need to create a Cypress Dashboard account for this to work, since we'll be using the Record Key provided by that service.

Setup

It's worth noting that I'm working with CraftCMS sites that use npm/node for preprocessing CSS/JS. So in addition to NPM dependencies I've also got Composer and all its PHP dependencies.

I'm also running everything in Docker containers locally, which are basically mirrored by Platform.sh's deployment containers.

We are using Platform.sh for hosting, which is a great, docker-based solution for hosting fullstack applications where you need a database, runtime environment, and a server.

Triggers, Environments, and Integrations

There are probably lots of posts about setting up Cypress. But it took a lot of work and a lot of coordination with the Cypress DevRel team (who are fantastic) and Platform.sh support to be able to get this whole thing hooked up. Hopefully this post helps some other poor sap tasked with setting this up. The process is fairly specific to Platform.sh for hosting, but other hosting providers may provide similar tools.

Essentially we have three separate systems that all need to communicate with each other. The PR is created by a developer in Github. Upon creation, we need to provision a new docker environment in Platform.sh with a dedicated URL. This takes a few minutes to build and propagate. Once it's ready, Cypress tests need to be triggered, but they also need the URL of the Platform.sh environment. Cypress also needs a runtime environment with Node.js installed to run the tests. And once those are complete, it needs to talk back to the Github PR.

So how do we make this work?

Platform.sh Integrations

The first step is pretty native to Platform.sh, but your hosting provider probably offers this as well. Using the Platfrom CLI you can create an integration that triggers a new environment on creation of every Pull Request in Github.

Platform.sh has great docs on this integration, so I'll just say this is the basic command:

platform integration:add --type=github --project=PLATFORMSH_PROJECT_ID --token=GITHUB-USER-TOKEN --repository=USER/REPOSITORY

And this is what my settings look like (found by running platform integration:get <integration_id>):

+---------------------------------+-------------------------------------------------------------------------------------+
| Property                        | Value                                                                               |
+---------------------------------+-------------------------------------------------------------------------------------+
| id                              | xxxxxxxx                                                                       |
| type                            | github                                                                              |
| base_url                        |                                                                                     |
| repository                      | <GH_Org_Name/GH_Repo_Name>                                                             |
| fetch_branches                  | true                                                                                |
| prune_branches                  | true                                                                                |
| build_pull_requests             | true                                                                                |
| build_draft_pull_requests       | false                                                                               |
| build_pull_requests_post_merge  | true                                                                                |
| pull_requests_clone_parent_data | true                                                                                |
| hook_url                        | <Platform_Generated_URL> |
+---------------------------------+-------------------------------------------------------------------------------------+

Once you have that built out - you're halfway there! And it's pretty easy...

Github Actions (and API)

Now that we have an environment created, let's create a Github Action to spin up a new linux environment with Node.js installed and run our Cypress tests from there.

I had never used Github Actions before so this took a bit work. And truthfully, there are a lot of Cypress examples out there that all point you in different directions and don't seem up to date with each other. So I'll provide exactly what worked for me.

In your repository, create a new directory called .github (if you don't already have one) and then within that directory another called workflows, and within that a file named cypress.yml. This file is where we will program our Action. Go ahead and paste in the following, being sure to modify your inputs.

This job will create a new environment using ubuntu 16.04, then npm install your dependencies, then cypress run all your tests.

# .github/workflows/cypress.yml 

name: Manually Triggered Cypress Tests with installation job

on:
  workflow_dispatch:
    inputs:
      baseUrl:
        description: 'Testing Domain'
        required: true
        default: 'https://www.uschamber.com'
      title:
        description: 'Merge Comment'
        required: true
        default: 'Running Cypress Tests'

jobs:
  cypress-run:
    runs-on: ubuntu-16.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      # Install NPM dependencies, cache them correctly
      # and run all Cypress tests
      - name: Cypress run
        uses: cypress-io/github-action@v2
        with:
          record: true
        env:
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          CYPRESS_baseUrl: ${{ github.event.inputs.baseUrl }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          COMMIT_INFO_MESSAGE: ${{ github.event.inputs.title }}

(see the gist here)

Notice the on section - this seems to be a little-known fact about Actions - you can actually pass variables in from a remote source. This is perfect for us because we want to give it the URL of the Pull Request environment, not our production URL!

Truthfully, I'm not sure we need the COMMIT_INFO_MESSAGE, but if I figure that out I'll update it later.

Also note that we are using some secrets. Secrets can be added to your repository under the Settings > Secrets UI. Grab your Cypress Record Key from Cypress Dashboard (log in at cypress.io for a free account), then drop it in as a secret called CYPRESS_RECORD_KEY. The GITHUB_TOKEN is automatically created by Github, so don't worry about that one.

Now that we've gotten this far, go ahead and trigger your script! You can hit it manually under the Actions Tab of your repository:

If it errors out, follow the error messages to hopefully debug your way out of the issue.

Platform.sh Activity Scripts

Now that we have a manual action set up we need a trigger. And of course, we don't want it triggered when the Pull Request is created, but instead once the environment is created.

Enter Activity Scripts. Platform.sh allows you to basically hook into the deploy process of its containers and run any JavaScript after the fact.

Quick note on these: The scripts only run ES5, and they are incredibly difficult to debug. I really hope Platform.sh updates them in the future to work with promises and ES6+. It would also be really nice to trigger them without pushing arbitrary code.

Rather than pasting the code here - I'll link to the gist that you can copy.

In order for this script to work you'll need to create a Github Token and store it in your Platform.sh environment variables as GITHUB_AUTH.

Github Token permissions for your reference:

To understand the activity script, you'll need to understand that Platform.sh has a different git strategy than Github.

This script gets the base URL from Platform.sh, grabs the Platform.sh branch name (pr-x) and then hits the Github Pull Request API to get the source branch name feature-x, then hits the Github Actions API and hits our cypress.yml action with the branch name and environment URL.

If you want to debug the script you can use the platform integration:log <integration_id> to get the latest run. It will show you anything that was printed to the stdout.

Cypress Dashboard

The final step in the whole strategy is to post the results of the test right back to the PR. Luckily, Cypress Dashboard handles this for us. Remember how we grabbed the RecordID earlier in the Action script?

Navigate to cypress.io and check your Project Settings. Fill out the Github Integrations section, and you're DONE.

Testing

Now give it a whirl! Open a new PR, watch the environment automatically build in Platform.sh, then check the log for the script to see it trigger Github Actions, then watch the actions run, then see it post a status and comment to your Github Pull Request.

This has helped our team go much faster! Hopefully it helps your team too.

Shout out to Kevin Old @ Cypress who proactively reached out to me on Twitter when I was at wits end and dealt with all my questions on making continuous testing a reality for us.