I'm gonna start with a puzzle for you...there are 3 light switches outside a room, one of the switches turns on the light inside the room, the other two do nothing. There is a door to get in to the room and you can only open it once, you can't flick any switches with the door open, but before opening the door you can flick as many switches as you like. So how do you figure out which switch turns on the light inside the room?
OK so we'll come back to that a little later. At the moment I'm reading a book called 'The Drunkard's Walk' by Leonard Mlodinow, subtitled How Randomness rules our lives. It's a really interesting book with some great anecdotes and examples for the lay person to understand the science he explains. Before I get on to the great examples here's why I chose to read the book. It was an Amazon recommendation. No, just kidding, I'm fascinated by how we see patterns that don't exist. E.g if we toss a coin 10 times and get heads, what is most likely to be the 11th time?.....in reality we often make assumptions based on what it has been; e.g. the previous 10 coin tosses have helped us to know the 11th coin toss. No it hasn't, not really. When you toss a coin you have a 50:50 chance of heads and tails. Eventually, as long as the coin is not weighted it will likely balance out, but that might take 100, 1,000, or even 100,000 coin tosses to get to that 50:50 ratio. The 11th coin toss is completely independent of the other 10. You might think that works against the following initial example but remember that we have no idea how many tosses we need to see to get a balanced result, and 10 coin tosses is potentially not enough.
Regression to the mean
This essentially means that "in any series of random events an extraordinary event is most likely to be followed, due partly by chance, by a more ordinary one". Mlodinow uses an aeronautical example but I'm going to provide a different one:
Consider a basketball player who on average is shooting 50% from the field, if he has a game where he shoots 60+% he will likely get praise from the coach.Another player averaging the same 50%, shoots <40% but he gets (let's call it..) 'less positive feedback'. Often in this situation in the next game the player receiving positive feedback shoots closer to their average, but so does the player receiving negative feedback. The coach giving positive feedback thinks; "great when I give positive feedback he get's worse". And the coach who gave negative feedback thinks "Jeez, when I chew this guy out, he performs better". In reality neither coach is right, the player's performance simply regressed to their average. If they performed particularly badly, or particularly well then next time out they were likely to perform closer to their average.
This is a phenomenon where someone with preconceived ideas/theory is more likely to see the evidence that supports their ideas/theory. I find this a common occurrence in sport science as well as every day life. And don't for a moment think that you don't have pre-conceived ideas, because we both know that you do. So here's the example that Mlodinow gives; Here is a sequence of numbers that follow a rule. 2, 4, 6. As the budding scientist you are trying to find the rule so you can test it by giving me a sequence of 3 numbers to check the rule, and I will tell you whether or not they fit the rule. Have a think about your sequence of numbers.......
...OK then, so you said 8, 10, 12.....or 18, 20, 22....or 16, 20, 24. And I tell you that yes, it fits the rule. So what's the rule? At this point I am betting (and you will probably deny it) that it was something to with even numbers or multiples of two. If this was your theory, then your test has proven the rule, right. So it the rule correct!? No. The rule is simply that each successive number in the sequence must be higher than the previous. You chose to test your hypothesis, you tested your theory, or preconceived ideas. And because it fitted you assumed you were right. If you had chosen to test your rule by looking for something that would have given a negative result to your theory e.g. 3, 6, 9, or 11, 12, 13, (which wouldn't have fitted in to your even number theory) then you would have found it did still fit the rule and had to ask more questions, or revise your hypothesis.
Back to the Puzzle
OK then so back to the puzzle we go. How did you get on? Did you get it?
So there are 3 light switches outside a room. And a light behind a closed door. So the solution;
Let's call them Light switch (LS) 1, 2, and 3. You flick LS1 and leave it a few minutes. Then you flick it back. And flick LS2 and open the door. if the light is on, then it was LS2, if the light is off then you know it was LS1 or LS3 right. Is the bulb hot? If so then we can pretty safely assume that it is LS1, if the bulb is cold then we have a good idea that it is LS3.
OK so that's not hugely relevant to the discussion of probability and randomness, my tenuous link is that perhaps your preconceived ideas of how you tell if a light switch works is built solely on whether the bulb is illuminated.
Anyway, now you can go try it on someone else.