We often have an inflated opinion of our understanding of reality. We think we know how things work. But as Philip Tetlock showed elegantly in his research, even experts are typically not much more accurate than anyone taking random guesses. This is bad because if your understanding of reality is inaccurate, you’ll inevitably make some horrible decisions in your life.
The main reason for our false understanding seems to be confirmation bias. Let’s say you make a prediction: ”Paul and Sarah will definitely get together”. Now if Paul and Sarah don’t get together, you’ll either find a nice excuse why you’re wrong, ”well, I couldn’t have known about X, and that changed everything”, or you’ll just slowly forget that you were wrong over time. It’s not pleasant being wrong, so why should you keep reminding yourself of it? If your prediction about Paul and Sarah is right, you’ll say that you knew it all along and congratulate yourself on your profound understanding of human nature.
Another example from University. We know that passive recall of information, just reading a textbook or watching a video, doesn’t work well for learning it. It feels different, though. When you’re reading that textbook for your chemistry exam again, you’ll be thinking ”yeah, I remember this part”. It’s the illusion of knowledge through recognition. You’ve seen it before, you remember parts, so you have the impression that you know this information. When the exam day arrives, and you get asked a question about the text, you draw a blank. You didn’t actually know the information, you just had a sort of de-ja vu moment while re-reading the text.
So, what can you do about it? How can you measure your understanding of something accurately?
The answer is that you measure it by how accurate your predictions are in a space. If you can predict what happens, you’ve thoroughly understood what factors influence an outcome and how they interrelate to each other. You know the cause and effect relationships and can already see at the beginning how things will turn out.
Of course, most people don’t put their predictions to the test. We’ve already established that with the two examples above. Often, they don’t even make one consciously. It’s easy to express an opinion. It’s hard to bet money or reputation on one and making it a prediction. Only then, the level of true certainty shows.
Take the blogger Scott Alexander, for instance, every year he and some of his friends, write a list of predictions for the coming year. He writes down a clear prediction (“Major flare-up (worse than in past 50 years) in China/Taiwan conflict”) and then puts a prediction of this happening expressed as a percentage next to it (e.g., he believes a major flare-up is only about 5% likely). At the end of the year, he evaluates how he did and grades himself.
It’s such an easy and effective thing to do. Just write down simple predictions for things you care about and check in later if they turned out that way. If you write down the reasoning, you’ll even learn where your thinking was flawed. With this practice, you can overcome your confirmation bias and laziness and gain a realistic estimation of your understanding of how reality works.