The love triangle between k6, Bitbucket pipelines, and New Relic

Aside from Cypress, I've been experimenting lately with k6, another Javascript-based open-source testing tool 🧐.

As I mentioned in my previous post, I've recently joined a startup, who's a leader in an exciting and maturing category in the DevOps world (separate story of course 😉). I've quickly established a UI and API automation framework using Cypress for only a few weeks and its a matter of up-skilling "Java/Selenium"-based Quality engineers on the coolest e2e tool in the market right now. However, the company is suddenly facing a growing pain of increasing Performance issues - hence, comes the newest saviour in town - k6 🦾.

Since I am still on the "Proof-of-concept" stage (as the company is currently using JMeter with a limited number of tests and it's just not reliable), I wanted to see if it works locally first.

After a few yada-yada a.k.a building a test script (or the quick and dirty way of converting the .HAR file to k6 - that's another topic to write about), I wanted to schedule the tests every half an hour during weekdays. This k6 documentation using cron is heaven-sent 😇. I set my cron job (crontab -e) with the line below:

*/30 8-20 * * 1-5 /Users/poponuts/k6/scripts/run-test.sh

... while my bash script contains the following:

#!/bin/bash
# run-test.sh
echo $PATH
/usr/local/bin/k6 run --out statsd /Users/poponuts/k6/tests/login.js

Wondering what the --out statsd line is for? Well, for starters, I'd like the metrics to appear in a nice shiny dashboard - New Relic it is, since we already use this on our tech stacks. And yes, k6 has an extensive documentation on how to do that too. This involves running Docker on your local machine. Also note that NR Insights Key is not the same as NR User Key (something you can create yourself) so you need to ask your New Relic administrator to provision you with one.

As part of me wanting to do a bit of Vertical slicing, I decided to see if it works using our company's CI tool, Bitbucket pipeline. Taking inspiration from a sample "k6+CircleCI" template I found on the net, I took a stab with the following bitbucket-pipeline.yaml file, which surprisingly, worked on my first try 😎:

image: loadimpact/k6:latest
options:
  max-time: 20

pipelines:
  default:
  packages
  - step:
      name: Run performance tests
      script:
        - k6 run ./k6/tests/login.js

The only limitation using Bitbucket pipeline is that it is not that flexible enough to customise scheduled runs (minimum is an hourly execution 👎). C'mon, Atlassian team, heaps of catching up on other modern CI tools! Even TeamCity 🤢 can do that!

Since I wanted to run it more frequently and I am using my work laptop everyday, anyways, then I decided to schedule the runs via the k6 and New Relic love team ❤️‍🔥. Since Atlassian charges based on the duration of your builds using Bitbucket pipeline and it does not display fancy dashboards, anyways, then I decided to use the CI only on an ad-hoc basis (a.k.a. maybe never in the next few months). Once we scale our performance tests where ideally, executions take at least 10 minutes... just imagine my boss giving the death stare (virtually) when he gets the invoice ❤️‍🩹.