ABC's of the Statistics Used in Codes

The whole issue of determining if an outcome is unlikely or expected, can be summed up in one word: counting. We need to be able to count how many possible outcomes there are, and how many of them would qualify as equally "interesting" to the outcome we observed.

Consider a busy street corner.

Scenario 1: By chance, you see Fred Smith, whom you haven't seen since childhood. Although you are surprised, this happens to most people sometime in their lives, so it is actually expected.

Scenario 2: You wake up one morning and say, I think I'm going to run into Fred Smith today; and it happens.

Notice that the same event (seeing Fred) occurred in both scenarios, but in the first case it is expected and in the second it is extraordinary - a "1 in a million" chance.

Keep in mind that two ELS's "meeting on a street corner" (occurring in a text near each other), have the same considerations.

Why is there such a difference in the two scenarios? Let's count interesting outcomes. For scenario 1:

(a) There may be 1,000 people from your childhood who would have been just as surprising to run into; and

(b) It would seem just as surprising if the chance meeting occurred on any of several thousand days in your life.

Putting (a) and (b) together, we see that there are 1,000 times several thousand equally interesting possibilities for scenario one, which is several million. Therefore, the event that occurred - seeing someone from childhood sometime in your life - had several million ways of coming to pass.

For scenario 2 there is only one person, not 1,000, that would make an interesting outcome. And only one day, today, not several thousand days. So we have only 1 way that the interesting outcome could happen.

The moral of the story is: if we state in advance what we are looking for, and we are right even a few times, we demonstrate something extraordinary is going on. If we don't do this, then even events that seem surprising can not really be measured, and are statistically meaningless.

Specifying the rules of the game in advance is known as using an "a-priori" protocol. It is the opposite of the Mickey Mouse method, "find first, explain later" - also known as shooting an arrow and then drawing a target around wherever it hits.

Back One Level
Back to Home Page