The one sentence summary
With the right approach it is possible to evaluate confidently the claims that surround us.
- The book contains ten rules for thinking differently about numbers, plus one golden rule. They are:
- Search your feelings: how does this make me feel, and why? Check your emotional reaction.
- Ponder your personal experience: take the worm’s eye view (personal) as well as the bird’s eye view (statistical).
- Avoid premature enumeration: don’t take numbers at face value – establish what is really being counted.
- Step back and enjoy the view: ask yourself: is that a big number? Put the claim into context and look for comparisons.
- Get the back story: look behind the statistics to find out where they came from.
- Ask who is missing: not all data is comprehensive. Would our view be different if we knew more?
- Demand transparency when the computer says no: what algorithm was used and how accurate and helpful is it? Without intelligent openness big datasets cannot be trusted.
- Don’t take statistical bedrock for granted: most official statistics can be trusted, but many others can’t.
- Remember that misinformation can be beautiful too: beware misleading graphs and charts – they can be designed to prove pretty much anything, so check that you understand what the axes actually mean. The smarter they are, the more suspicious you should be.
- Keep an open mind: how might I be mistaken, and have the facts changes since?
+. The golden rule. Be curious: look deeper and ask questions.
WHAT’S GOOD ABOUT IT
- Good statistics are not a trick and they have the power to change the world for the better. But bad research and misplaced motivation can take us deep into a world of disinformation and obfuscation.
- Motivated reasoning is when someone’s judgement is strongly influenced by what they hope will be true.
- Some studies are subject to HARKing – Hypothesising After Results Known. A lot of contrary findings are never even published.
- A statistical metric may be a decent proxy for something that matters, but once you use that as a target to be improved, it will be distorted, faked or undermined.
- Keynes is reported to have said: “When my information changes, I alter my conclusions. What do you do, sir?” There’s no firm evidence that he did, but the quote is useful for being prepared to change your views.
- Statistics in themselves don’t deliver benefits, but the use of statistics can.
- Actively open-minded thinkers make the best forecasters because they don’t cling too tightly to a single approach.
- The illusion of explanatory depth is a curiosity killer and a trap – if we think we already understand, why go deeper?
WHAT YOU HAVE TO WATCH
- Nothing. This is well-argued and should be helpful to all.