We all love information. The cliché of our era is that we’re all information addicts, drinking it straight from the tap during most of our waking hours. Since the dawn of 24-hour cable news we’ve grown accustomed to always having a stream of information available to us. The proliferation of social information from friends, family, people that we’ve never met and people we would never want to meet is the market’s response to our thirst to always know more.
We’re told that this trend will have dire consequences for our ability to focus, for our attention spans, for our ability to communicate with one another. I’m not necessarily convinced by these arguments, any significant technological change will lead to changes in the way we think and interact with one another, and I tend to lack the nostalgia for “how it used to be.” That said, what worries me is how this affects our decision making, especially at the upper levels of government and business.
The assumption today is that the data needed to make a decision is available, no matter what the question. And given the advances in computing and our skills in data analysis, it is easy to assume that a quantifiable answer to any question is always out there, it just needs to be found. This leads to a decision-making process which is focused on reducing or eliminating uncertainty or, more accurately, apparent uncertainty. We want a model which will calculate the consequences of our decisions out to the sixth decimal. We want to know when and where the effect will take place and we want to know how much it will cost. Of course, these are all appropriate questions to be asking. There’s only one problem: in the vast majority of cases, we can’t know any of this.
Any of these predictions would have to be the result of some sort of model. Broadly speaking, a model is a set of variables, definitions and piles of data which a computer would then combine according to a specific set of instructions, producing some output which then need to be interpreted and communicated by the people responsible for the model.
By definition, a model is a simplification of reality. While many models are extremely powerful — certain must be counted amongst the most impressive feats of our scientific and technological abilities — they are still a limited description of the processes which make up reality. Until there is a Matrix-like alternative reality where every physical, chemical and human interaction is accurately described and recorded, we will need to rely on models which can only offer an approximate vision of the real world.
When you speak with the people responsible for running the models and interpreting their results, the honest ones — and the vast majority are — will emphasize the inherent uncertainty of models’ output. While the results of the model are useful as an approximation, they are not a certain vision of a future condition. These modelers will emphasise that a model’s output will be influenced by not only the quality of the data which is used as inputs and the assumptions used in the design of the model, but also the model designer’s school of thought. Take the models which attempt to explain and predict international economic trends — what the cool kids call “macro-econometric models”. The fundamental disagreements among economists — should a government increase or decrease spending in a recession? — are reproduced in the design of the model. Being told that an econometric model predicts some outcome is not enough, we must also know which model was used, how it was designed and what data was used as inputs.
As a result of this, models must be used carefully. They are an extremely useful tool, but which have inherent limitations and expecting them to provide The Answer is a dangerous approach to decision making. If your decision making process is based on the perfect accuracy and predictive power of models, then you are setting yourself up for potentially dangerous results.
Knowing this, the decision-making process itself must be adapted and designed to be resilient to uncertainty. This does not mean avoiding the use of models and other predictive tools all together, but rather understanding that their usefulness has limits. Rather than saying that a particular policy will result in a drop in the unemployment rate of 2%, the possible outcomes need to be understood as a range. If the final result is 1.5%, it does not mean that the policy was a failure. It simply recognises the inherent margin of error in our predictive abilities.
This is not a popular message. Uncertainty is necessarily uncomfortable, we are built to want to know and to always want to know more. Admitting that we don’t know is difficult, especially when your decisions can have an impact on millions of people and costs in the billions of dollars. But building a facade of apparent certainty, of supposed knowledge, is even worse. While it may be comforting to think that we know more, the results will almost always be worse.
Decision makers have a double responsibility with regard to uncertainty. The first is being aware of the degree of uncertainty present in any decision. The second is building a decision making process which is resilient to such uncertainty.
