Political Scientists Aren’t: Their Forecasts Are Worse Than Random
There is a stubborn logical fallacy at work in the world of political forecasting, and that is a belief in the inherent determinism of the universe. As Karl Popper wrote in The Open Universe, scientific determinism is
the doctrine that the structure of the world is such that any event can be rationally predicted, with any desired degree of precision, if we are given a sufficiently precise description of past events, together with all the laws of nature.
Determinism has been pretty well debunked, except in fields like celestial mechanics, where long-term projections of the movements of planets and stars is possible because of the orderliness of the systems involved. But our more modern investigations into complexity and chaos have revealed that many systems are deeply and profoundly unpredictable. Most suitably interesting systems are subject to high degrees of randomness and their state at some point in the future is highly dependent on their state in the past, which is probably too complex to measure.
But that doesn’t stop political ‘scientists’ from persisting in prediction:
Jacqueline Stephens, Political Scientists Are Lousy Forecasters via NYTimes.com
[…] in the 1980s, the political psychologist Philip E. Tetlock began systematically quizzing 284 political experts — most of whom were political science Ph.D.’s — on dozens of basic questions, like whether a country would go to war, leave NATO or change its boundaries or a political leader would remain in office. His book “Expert Political Judgment: How Good Is It? How Can We Know?” won the A.P.S.A.’s prize for the best book published on government, politics or international affairs.
Professor Tetlock’s main finding? Chimps randomly throwing darts at the possible outcomes would have done almost as well as the experts.
These results wouldn’t surprise the guru of the scientific method, Karl Popper, whose 1934 book “The Logic of Scientific Discovery” remains the cornerstone of the scientific method. Yet Mr. Popper himself scoffed at the pretensions of the social sciences: “Long-term prophecies can be derived from scientific conditional predictions only if they apply to systems which can be described as well-isolated, stationary, and recurrent. These systems are very rare in nature; and modern society is not one of them.”
Some political scientist have started to pay attention to the forms and norms of so-called future studies, and to consider alternative scenarios for political outcomes, rather than attempting to apply deductive or inductive logic to predict the future.
Personally, I find hope in abductive reasoning or inference, or as it first proponent, Charles Sanders Pierce, called it, ‘guessing’. Here’s a fairly concise definition from the Ohio State University Laborartory for Artificial Intelligence Research:
Abductive InferenceAbduction or Inference to the Best Explanation is a form of inference that follows a pattern like this:
D is a collection of data (facts, observations, givens),
H explains D (would, if true, explain D),
No other hypothesis explains D as well as H does.
————————————————————————————
Therefore, H is probably correct.The strength of an abductive conclusion will in general depend on several factors, including:
- how good H is by itself, independently of considering the alternatives,
- how decisevely H surpasses the alternatives,
- how thorough the search was for alternative explanations, and
- pragmatic considerations, including
- the costs of being wrong and the benefits of being right,
- how strong the need is to come to a conclusion at all, especially considering the possibility of seeking further evidence before deciding.
That the strength of abductive conclusion ‘will in general’ depend on these factors means that it should depend on these factors, and that insofar as we are intelligent creatures, our conclusions will actually depend on these factors.
And the creation of scenarios to consider possible futures can be approached as the creation of different sets of data, D, in the formulation above, and the attempt to come up with the most plausible hypothesis, H, to explain how we might have arrived at the corresponding data set.
In a way this can be thought of as generating theories to explain the data, in a circular reasoning process.
My feeling is that most economists are grounded in rules they are trying to prove, and are operating deductively or inductively, inferring the general from the specific or the specific from the general. Abduction can be considered reasoning from the consequent to the antecedent, from outcomes to precipitating causes.
It’s worth noting that there is no clockwork mechanism here, no guaranteed success in abduction: it’s guessing, after all.
And for the purposes of future studies — conjecturing hypothetical situations in the future and possible ways that events might have unfolded to reach those states — it is doubly likely for a guess to unravel. Add to the mix the growing uncertainty of the post-normal world, characterized by VUCA — volatility, uncertainty, complexity and ambiguity — and abduction is stretched even thinner.
But I still believe it’s the only tool we have that extends our thinking, instead of simply applying what we already know. Abduction is generative — it creates theories — instead of simply reusing existing rules and case reasoning. Let’s hope the political scientists get the memo.
