How to choose a median result when running Google Lighthouse multiple times

According to the docs, running Google Lighthouse five times against the same URL will reduce the variability of your results by around half if you pick the median result.

The problem is, how do you pick the right result to use as your median, or representative result?

A naïve approach would be to take the median of the performance score, however since the performance score itself is a weighted average of other scores, and is more prone to outliers, it isn’t as useful as you’d think.

Lighthouse actually exports a function to help you calculate which result to use: computeMedianRun.

How to use computeMedianRun in Node

const spawnSync = require('child_process').spawnSync;
const lighthouseCli = require.resolve('lighthouse/lighthouse-cli');
const {
  computeMedianRun,
} = require('lighthouse/lighthouse-core/lib/median-run.js');

const results = [];
for (let i = 0; i < 5; i++) {
  console.log(`Running Lighthouse attempt #${i + 1}...`);
  const { status = -1, stdout } = spawnSync('node', [
    lighthouseCli,
    'https://example.com',
    '--output=json',
  ]);
  if (status !== 0) {
    console.log('Lighthouse failed, skipping run...');
    continue;
  }
  results.push(JSON.parse(stdout));
}

const median = computeMedianRun(results);
console.log(
  'Median performance score was',
  median.categories.performance.score * 100
);

Under the hood, computeMedianRun finds the run that’s closest to the median First Contentful Paint (FCP), AND the closest to the median Time to Interactive (TTI).

FCP and TTI are used because they’re the earliest and latest moments in a page’s lifecycle.

(This is an article posted to my blog at maxrozen.com. You can read it online by clicking here.)