I agree in many cases, but I think in other cases one can hypothesize that the outcome isn't going to meaningfully different and eyeballing the problem or working from first principles is fine.
We A/B in order to avoid arguing over a series of potentially horizontal changes. We don't want to make a change that nobody can agree on is an actual improvement. In other scenarios, the outcomes are too chaotic to justify the time it would take to set up a truly scientific test. It's case by case basis, but hopefully the conversation can happen among people who all understand the tradeoffs.
Respect the A/B test, understand the non-universality of it as a solution and potential local maxima problems. 😄
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
I agree in many cases, but I think in other cases one can hypothesize that the outcome isn't going to meaningfully different and eyeballing the problem or working from first principles is fine.
We A/B in order to avoid arguing over a series of potentially horizontal changes. We don't want to make a change that nobody can agree on is an actual improvement. In other scenarios, the outcomes are too chaotic to justify the time it would take to set up a truly scientific test. It's case by case basis, but hopefully the conversation can happen among people who all understand the tradeoffs.
Respect the A/B test, understand the non-universality of it as a solution and potential local maxima problems. 😄