In Part 1, we covered the concept of generators. In Part 2, we added a runner on top of them. In Part 3, we brought shrinking capabilities to all our generators.
But at this point, our runner is not using shrinker capabilities we introduced in previous part.
Shrinkers have been introduced in an attempt to find the smallest failing case. They achieve one main purpose: given one value they compute strictly smaller ones.
From a runner perspective, in order to get the smallest possible failing case we want to shrink as soon as we find an issue. So we generate values and run the predicate on it and as soon as we find an issue we take this value and shrink it. Moreover while iterating over those shrunk values we may find another failing one. So, we shrink it and iterate over it until we reach the smallest failing case (a failing value for which none of its shrunk values can cause a failure).
The process is summarized into the diagram below:
In terms of code, we can adapt our miniFc.assert as follow:
miniFc.property=(generator,predicate)=>{return{generate(mrng){returngenerator.generate(mrng);},shrink(value){returngenerator.shrink(value);},run(valueUnderTest){returnpredicate(valueUnderTest);}}}functionexecuteAndShrink(valueUnderTest,property){if (!property.run(valueUnderTest)){for (constshrunkValueofproperty.shrink(valueUnderTest)){constshrunkResults=executeAndShrink(shrunkValue,property);if (shrunkResults.failed){returnshrunkResults;}}return{failed:true,value:valueUnderTest};}return{failed:false};}miniFc.assert=(property,{seed=Date.now()}={})=>{letrng=prand.xoroshiro128plus(seed);for (letrunId=0;runId!==100;++runId){constvalueUnderTest=property.generate(newRandom(rng));consttestResults=executeAndShrink(valueUnderTest,property);if (testResults.failed){thrownewError(`Property failed after ${runId+1} runs with value ${JSON.stringify(testResults.value)} (seed: ${seed})`);}rng=rng.jump();}}
Here we are, our home-made property based framework is now able to generate values, run tests and find failures, and in case of failure to shrink it towards something smaller to help our users.
Our way to map from a generator to another requires both a map and an un-map function. Unfortunately, reversing an entity is not always feasible and may require an extra work just to get shrinking capabilities
Our way to map from a generator makes it difficult to provide a oneof generator with shrinking capabilities
Current generators are too dummy, they miss lots of corner cases. While uniform random is good, it may takes an infinite time to find bugs…
I hope this serie of articles helpt you to get a better understanding of how property based testing may work under-the-hood. The minimal framework we finally got was how fast-check looked like during my first experiments, but no version has been released with this design.
Top comments (0)
Subscribe
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)