Let's assume following source code generating random number with Math.random() and faker.random.number():
var faker = require('faker');
faker.seed(1);
console.log(
"Faker goes:",
faker.random.number({ max: 100 })
);
console.log(
"Math.random() goes:",
Math.random() * 100
);
And now let's try to run it several times in a row:
❯ node index.js
Faker goes: 42
Math.random() goes: 24.270154608840078
❯ node index.js
Faker goes: 42
Math.random() goes: 17.379030134115037
❯ node index.js
Faker goes: 42
Math.random() goes: 66.8433058100395
One quite common thing to do while writing tests is to write down what data got generated during the test run and assert against that value:
let myRandomInt = faker.random.number({ max: 100 });
// let myRandomInt = Math.random() * 100;
assert.equals(myRandomInt, 42, 'Coincidence? I think not');
Now while there are better methods to write tests, this is a quick win and when done right can work quite well. As you can see from the values generated above when you use Math.random()
you will get different result every time. While when using faker
the results seem stable.
Faker will always give you the same results when all the calls to it are exactly the same till your call. The problem arises when you for some reason add another call to faker
before your call:
faker.random.number(); // Extra faker call
let myRandomInt = faker.random.number({ max: 100 });
assert.equals( // This will fail ☹️
myRandomInt,
42,
'Coincidence? I think not'
);
How to solve this? Using seed, which will reset the pseudo-random sequence:
faker.random.number(); // Extra faker call
faker.seed(1);
let myRandomInt = faker.random.number({ max: 100 });
assert.equals( // It works again ✨
myRandomInt,
42,
'Coincidence? I think not'
);
Conclusion
If you want to hard-code assert expected values in your test, you should make sure that you:
- In your tests use stable random generator like
faker
insteadMath.random()
. - Pin
faker.seed(x)
to a constant value before generating data for each test.
Photo by Riho Kroll on Unsplash
Top comments (2)
Good summary! Curious, under what circumstances would you want to generate seeded random values in a test?
(As opposed to just explicitly stubbing
Math.random()
to return43
, for example.)Great question. I would say that you actually always want to have a pseudo-random seed (good trick is to pin it to pull-request branch name), because this gives you the ability to "shake the universe". Which turns out tend to uncover a lot of edge-cases your manual dice roll did not account for. On each run of the CI you are more and more sure that every possible input combination is taken care of without writing additional test code.
I would go as far as stating that an ember app (that's what I mostly work with) of any non-trivial size/complexity that does not use pseudo-random seed in the tests for running ember-exam has at least one hidden race condition.
I had few posts touching this topic (1), (2)