DEV Community

Discussion on: How important is stats knowledge in software development?

Collapse
 
evanoman profile image
Evan Oman • Edited

Average is really not a valid aggregation function in most cases (Digital Ocean metrics I'm staring right at you)

Great point, in fact this demonstration from AutoDesk shows that several common aggregate functions can easily be tricked.

GIF
(here all of the plots have the same summary stats, despite having very different shapes)

The lesson? Always plot your data!

Collapse
 
lepinekong profile image
lepinekong

Average is rarely a good metrics when law is not normal which is very often the case. Median which is even simpler to calculate is a better approximator, it seems traders on Wallstreet because that's one of their preferred indicator ;)

Thread Thread
 
xowap profile image
Rémy 🤖

If I follow you, since all distributions are normal you just need the average to understand your data? Because it seems to me that the normal function takes two parameters (average and standard deviation). With just one of those parameters you can reach the conclusion that your data is maybe centered on that average if the distribution happens to be normal.

Now suppose that instead of considering that traders on Wall Street use averages we look at the fact that they all wear navy socks. What does this tell us? That Wall Street traders wear navy socks and possibly other people too.

While your business might be dependent on knowing that it is possible that some people wear navy socks, I highly doubt that it yields any practical value.

All statistics are here to answer a specific question. If there is no question you might as well put 42 everywhere.

To give an example, you could ask yourself if your website is lagging. Then the answer would be to look at the 99th percentile of response generation time and to make sure that no requests take longer than an acceptable amount of time.

And very often, the answer is not an average. It's something specific to the business question you're asking. I don't doubt that Wall Street uses averages but the real question is to know what they do with it.

Thread Thread
 
lepinekong profile image
lepinekong • Edited

"All distributions follow normal"?!!! absolutely NOT that's why AVERAGE IS NOT USUALLY A GOOD METRICS or at least you should consider multiple samples like in a method called "Statistical Process Control" (I've been specialist of that in the past)

So Traders (I've been semi-professional trader's analyst for a bank 20 years ago ) do NOT USE AVERAGE for assessing what they call horizontal support and resistance, they use average only for mobile indicators.

Thread Thread
 
lepinekong profile image
lepinekong

It's funny people don't even read before answering ;)