The Law of Large Numbers

How to think about events in such a way that they can be counted and a decision can be made about how much data is enough.

What is Law of Large numbers????

Let us understand with an example.

Consider a city with two hospitals. One, with about 15 births per day and one with about 45 births per day. There are about 50% of babies born who were boys and 50% who were girls. So the probability is 50%.

Which of these hospitals, do you think has more days when 60% or more of the babies born are boys -- the hospital with 15 births per day or the hospital with 45 births per day?

Most people think there would be no difference. That there would be the same number of days in a year at the 15 birth per day hospital and the 45 birth per day hospital, when 60% or more of the babies born are boys.

But that is not true.

To understand the difference, we need to know Law of large numbers.
Consider hospital with 15 births. There's nine boys and six girls, so that's 60% boys.

Consider hospital with 45 births per day. That's 27 boys born and 18 girls born, that's 60% boys.

But isn't hospital with 45 births with 60% boys an unusual pattern to get, given that we know that there's a 50/50 ratio of boys and girls. Drawing from a 50/50 distribution, if you got 27 boys and 18 girls that's only going to happen 3 in 100 times.

It could happen, but it is pretty darned unlikely.

Now, what's going on here?

The principal that we use to understand this is the Law of large numbers, which says that, sample values, for example, proportions, resemble population values as a function of their size.

The larger the sample, the less likely it is that you will get a fluke, a very unrepresentative value. So 60% is a very unrepresentative value for 50%, which is close to the true value. But it's common to get that kind of difference with a small sample. If the sample gets large enough it becomes virtually impossible.

So you're going to get 60% or more boys at a hospital with 15 births every few days. You'll get 60% or more boys at a hospital with 45 births, maybe nine or ten times a year.

Let us consider the example of hiring for a job. Suppose you interview a man with a great record, a terrific recommendation from his previous employers, but in the interview the fellow didn't do great. We generally tend to reject that guy.

That sounds like the kind of thing that happens all the time, right? But is the judgement really a reasonable one?

I don't want to tell my opinion, instead let us see what law of large numbers or in general Statistics tells about this.

To help you think about that, suppose a soccer coach is looking for a striker, and he goes to a practice for a high school kid, who has a great scoring record and terrific reviews from his coaches. But at this practice, the guy misses some easy points and he just doesn't seem in control of the ball. So if coach thinks that the kid shouldn't be pursued.

Now, is the coach's judgment reasonable or not?

People who know sports are quite likely to say no that's really not so reasonable.

One practice, it's just not that much evidence, there's lots of variability. Kid could have an off day or any other reason.

Considering this scenario let's go back to hiring example. Do you still think decision is reasonable?

Well, the 30 minute unstructured interview is not that much evidence.
In fact, people have looked at how well you can predict performance in college, to see how well that interview rating predicts performance. And the correlation almost never exceeds 0.10, that's very, very small.

That's equivalent to increasing the likelihood of hiring the better of two candidates from 50/50, which is what you would get if you were going to flip a coin to make the decision, to a 53% chance.

If you have past performance and other judgments by other people, you can do quite well in predicting these same kinds of performance that I just mentioned. In fact, if you weight things properly that are in the folder, you can raise the chances of picking the right person to 65% or 75%. So this is the principle of law of large numbers.

To say that sample values for events having a chance component resemble population values for those events as a function of their size. In fact, the law of large numbers only applies where there is some kind of a chance component. There's no chance component to your measurement of how far it is between two cities, and the law of large numbers is irrelevant there. One measurement will do you, but sports or job interview are very variable.

We know that performance can be much higher or much lower on any given occasion. But most people never observe all that many interviews. And you don't get to see how well the prediction from the interview corresponds to performance of the person on the job.

The truth is the employer's judgment is even worse than the coach's judgement because interviews are not a sample of job performance or school performance. They're a sample of interview performance.

Interviews and performance on the job require different skills.

So the law of large numbers applies to all kinds of events in everyday life.

We apply the law of large numbers when we see the variability, the error, but not for events that are just as important where we don't. The principle, here, is if your variable is human behavior, assume there is error variance, and adjust your judgement accordingly.

Credit: Thanks to Richard E. Nisbett for Mindware: Critical Thinking for the Information Age course on Coursera.