A/B testing is, in my experience, usually an attempt to improve some business metric like landing page conversions and engagement etc. They're great if done right, but they inherently add a lot of complexity and coordination that can cause holes that are hard to dig out of.
Big orgs do a lot of A/B testing, but they devote the resources to making it work, and I'm sure they still have their issues. Smaller engineering orgs get in trouble trying to add split testing into an already rigorous workload.
There is also a stats issue involved. And how many misunderstand stats.
Seen way too many startup spending way too much time trying to use A/B, as a means to try find a magical 10x conversion rate improvement through it. With sample size of under a thousand.
Over simplifying, my rule of thumb, until one reach over a million hits per month, or a team of 50, one should never consider A/B. Till then incremental improvement and feedback is good enough.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
As a super beginner dev, I'd like to know a little bit more about why A/B testing might qualify as a practice that leads to tech debt.
It seems like something that giants like Google and FB do all the time. Is A/B testing not a practice intended to prevent tech debt?
A/B testing is, in my experience, usually an attempt to improve some business metric like landing page conversions and engagement etc. They're great if done right, but they inherently add a lot of complexity and coordination that can cause holes that are hard to dig out of.
Big orgs do a lot of A/B testing, but they devote the resources to making it work, and I'm sure they still have their issues. Smaller engineering orgs get in trouble trying to add split testing into an already rigorous workload.
There is also a stats issue involved. And how many misunderstand stats.
Seen way too many startup spending way too much time trying to use A/B, as a means to try find a magical 10x conversion rate improvement through it. With sample size of under a thousand.
Over simplifying, my rule of thumb, until one reach over a million hits per month, or a team of 50, one should never consider A/B. Till then incremental improvement and feedback is good enough.