DEV Community

Cover image for How to choose a median result when running Google Lighthouse multiple times
Max Rozen
Max Rozen

Posted on • Originally published at maxrozen.com on

How to choose a median result when running Google Lighthouse multiple times

According to the docs, running Google Lighthouse five times against the same URL will reduce the variability of your results by around half if you pick the median result.

The problem is, how do you pick the right result to use as your median, or representative result?

A naïve approach would be to take the median of the performance score, however since the performance score itself is a weighted average of other scores, and is more prone to outliers, it isn’t as useful as you’d think.

Lighthouse actually exports a function to help you calculate which result to use: computeMedianRun.

How to use computeMedianRun in Node

const spawnSync = require('child_process').spawnSync;
const lighthouseCli = require.resolve('lighthouse/lighthouse-cli');
const {
  computeMedianRun,
} = require('lighthouse/lighthouse-core/lib/median-run.js');

const results = [];
for (let i = 0; i < 5; i++) {
  console.log(`Running Lighthouse attempt #${i + 1}...`);
  const { status = -1, stdout } = spawnSync('node', [
    lighthouseCli,
    'https://example.com',
    '--output=json',
  ]);
  if (status !== 0) {
    console.log('Lighthouse failed, skipping run...');
    continue;
  }
  results.push(JSON.parse(stdout));
}

const median = computeMedianRun(results);
console.log(
  'Median performance score was',
  median.categories.performance.score * 100
);
Enter fullscreen mode Exit fullscreen mode

Under the hood, computeMedianRun finds the run that’s closest to the median First Contentful Paint (FCP), AND the closest to the median Time to Interactive (TTI).

FCP and TTI are used because they’re the earliest and latest moments in a page’s lifecycle.

(This is an article posted to my blog at maxrozen.com. You can read it online by clicking here.)

Top comments (1)

Collapse
 
pavelloz profile image
Paweł Kowalski

Very cool! :D