If the last two years have taught us anything it is that we can’t always predict future events. But to move ahead we all need to make decisions without certainty. How can we make better decisions in these uncertain times?
The Superforecasters
In 2011, IARPA (a division of the US Intelligence Agency) launched a competition to find better methods for forecasting geo-political events. At the time Dr. Phil Tetlock was working at the University of Pennsylvania and entered a team with colleague Dr. Barbara Mellers (now his wife). They called their team the Good Judgement Project (GJP) and relied heavily on volunteers and the so-called wisdom of crowds.
The IARPA contest ran for four years asking contestant teams to give their predictions for over world 500 events. At the competition’s conclusion, the GJP team emerged as the winner.
Phil’s work on prediction and decision science continued during those years and the years that followed. His research goal was to unlock the traits and characteristics of what he calls the Superforecasters – people who seem to consistently predict future events better than average. He shared his findings in his 2015 bestselling book, “Superforecasting: The Art and Science of Prediction.”
Phil studied thousands of forecasters, their predictions and the actual outcomes. He was surprised by what he found. He discovered that so-called expert predictions weren’t much better than chance (i.e., “dart-throwing chimps,” to use Phil’s words).
He also discovered that the best predictions didn’t come from massive super-computers or arcane “crystal ball” processes. Rather Phil and his coauthor Dan Gardner showed that Superforecasters all shared the following traits:- Gathering evidence from a variety of sources.
- Thinking probabilistically and specifically (which Annie Duke told us in Season 4 of Unleashed).
- Working in teams, especially ones with people who have different backgrounds and views.
- Keeping score and being willing to admit error. Constantly testing and adjusting.
When it comes to forecasting the future Phil says, “the right analogy here is not Nostradamus and prophesy. The right analogy is optometry. The increasing clarity that comes when we get a good optometrist, and the right glasses, boosts foresight from about 50% to 80%.”
He notes that the very best forecasters in the world assign 72-76% likelihood to things that did happen and 24-28% to things that didn’t. “This is not god-like omniscience.”
Superforecasters change their minds a lot and tend to make predictions tentatively. They experiment and change course often. They are followers of Adam Grant’s principle of rethinking.
"The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test."
-Phil Tetlock
Making Better Decisions
The problem business leaders and teams often face is that in our culture observers see probability thinking as weak or indecisive. Great leaders should be 100% certain of what is going to happen, shouldn’t they? They’re meant to have certainty and commitment. They have the best information, the most experience, and access to all the information. Isn’t that what followers are looking for? Even though these may be true there is still a lot of uncertainty when it comes to predicting the future.
Phil asks a very profound question: “What would our society look like, what would our debates look like, if instead of making categorical claims like ‘yes’ or ‘no’, or ‘we should do this’ and ‘shouldn’t do that’, if instead we got in the habit of making claims in more nuanced and granular ways?”
Employing the techniques of Superforcasters is part of the surprise-proofing playbook for leaders and organizations. Business decisions should be taken with:
- Assumptions and data documented.
- Predictions about the future assigned probabilities.
- Actions to mitigate even low probability events if the impact is relatively large and the cost to mitigate is low (e.g., lightning rods on buildings).
- Review of the quality of the decision-making outcomes AND the process for arriving at those decisions.
Thriving in an Uncertain World
There’s no question that our rapidly changing business environment is here to stay, and as leaders we need to make decisions without perfect foresight. That is why creating a culture of engagement and openness where information flows and it is OK not to pretend to have all the answers gives us a leg up on the competition.
Making better decisions using the techniques described here is a great step to designing a much more predictable business.
If you would like to learn more about creating a winning culture and surprise proofing your business, reach out to us for a chat.