That's a good question, similar to concerns I've heard before!
If the values that are important for your test are randomised, then of course your test may flip (unless your test dies some kind of "double accounting, calculating a result via a different method).
The point here is that in a test like "when a customer places a delivery order before a certain time, she should get a discount", then the other data points shouldn't matter. If a test flops because of a random data value that shouldn't have mattered, then it helps to find a problen with the production code. I've experienced this exact scenario - of course debugging it requires running the test until failure with lots of logging, but it's saved me in the long run.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
That's a good question, similar to concerns I've heard before!
If the values that are important for your test are randomised, then of course your test may flip (unless your test dies some kind of "double accounting, calculating a result via a different method).
The point here is that in a test like "when a customer places a delivery order before a certain time, she should get a discount", then the other data points shouldn't matter. If a test flops because of a random data value that shouldn't have mattered, then it helps to find a problen with the production code. I've experienced this exact scenario - of course debugging it requires running the test until failure with lots of logging, but it's saved me in the long run.