One of the features of ember-exam is that it will allow you to run your tests in random order. Which is great to uncover code that is leaking state in your application or tests.
Leaking state is tricky to uncover because it will stay hidden for most of the time, until something changes. In my case the change is usually caused by different order of tests being run. And therefore the state of one component / test starts leaking into other test run.
My primitive strategy is to take the test that started $suddenlyFailing
and look at all tests that ran before. One of them has to be the offender. This creates our initial $offendingList
.
The nice way to find the order of tests is to print to the console when one is starting / finishing. To do that, just add to your tests/test-helper.js
:
import Application from '../app';
import config from '../config/environment';
import { setApplication } from '@ember/test-helpers';
import QUnit from 'qunit';
import { start } from 'ember-qunit';
setApplication(Application.create(config.APP));
// START custom code
QUnit.on( 'testStart', function( data ) {
console.log( 'Start: ' + data.fullName );
} );
QUnit.on( 'testEnd', function( data ) {
console.log( 'End: ' + data.fullName + ' ---- ' + data.status );
} );
// END custom code
start();
As a bonus exercise you can run ember exam --random=[SEED]
several times for different values of SEED
and remove from the $offendingList
all the tests:
- that ran before
$suddenlyFailing
in the case when our test did not fail - that ran after
$suddenlyFailing
in the case when our test did fail
Top comments (0)