The extreme response of many governments to COVID-19 was driven by… computer models. We’re “following the science“, they told us (wherever it happens to be going today). “Experts say” they all say. What are you, a pinhead? Unfortunately the models turned out to have been a load of dingos’ kidneys, with significant consequences not just for the climate debate but for public faith in our institutions scientific and political. To limit the damage Benny Peiser calls for making a “Red Team” approach to science habitual, in which devils’ advocates are paid to poke holes in hot new research, question supposed consensus, critique standard models and challenge anything else that’s going to be used to silence criticism, override democratic mechanisms and impose drastic measures in haste based on fake certainty.
Some may argue that the pandemic models were actually good, because we responded to the alarming scenarios warning what would happen if we did nothing. Unfortunately the models also had quarantine scenarios and they didn’t come true either. In some cases the differences are so big as to be laughable despite their tragic economic and social consequences. Like New Zealand being predicted to have 14,000 coronavirus deaths and having 19.
The models were often poorly constructed and nobody thought to look. Thus on his “Rational Optimist” blog Matt Ridley gives them a thorough pounding starting with “Professor Neil Ferguson of Imperial College “stepped back” from the Sage group advising ministers when his lockdown-busting romantic trysts were exposed. Perhaps he should have been dropped for a more consequential misstep. Details of the model his team built to predict the epidemic are emerging and they are not pretty. In the respective words of four experienced modellers, the code is “deeply riddled” with bugs, “a fairly arbitrary Heath Robinson machine”, has “huge blocks of code – bad practice” and is “quite possibly the worst production code I have ever seen”.”
Interestingly, the same Prof. Ferguson already had a surprisingly spectacular history of failed disaster predictions: He said there’d be hundreds of thousands of deaths from mad cow disease, tens of thousands from swine flu and hundreds of millions from bird flu. Instead, Ridley notes tersely, “The final death toll in each case was in the hundreds.” Yet this dubious track record, with its serious real-world consequences, did not make Ferguson unpopular with the “experts say” crowd. In the contrary, he was treated as authoritative this time as well. And his new program, whose code was apparently altered before being released, predicted 40,000 deaths in Sweden by May 1, off by a factor of 15.
The problem with models is not that Neil Ferguson is a nitwit even if this particular one was a shambles. More fundamentally it’s that we expect too much of them and too often. Four centuries ago Galileo uttered one of the foundational maxims of modernity: that mathematics is the language in which the universe is written. Ever since, we’ve sought increasing precision in algebraic representations and predictions not just of chemistry or optics but of the economy, social behaviour and yes weather. And despite major advances in the hard sciences over centuries, including computers now being scarily better than the best humans at games like chess that we once thought relied on a kind of intuition and judgement machines could not exhibit, we’ve continued to trust this approach in other areas where the results have been distinctly unimpressive. Like predicting the stock market. In fact mathematizing the humanities into “social sciences” has arguably moved us backward not forward. Little is now heard of “cliometrics.” And the models have proved unhelpful, deceptively so in fact, even on non-linear processes in the hard sciences.
As Anthony Watts reminds us, not for the first time, Judith Curry talked about the “uncertainty monster” back in 2011. There is just too much we don’t know, about the data and about the relationships between various factors. But the public and the elite expect mathematical certainty and indeed demand it even though it’s not available. And when they don’t get it, they don’t seem to notice, on everything from disease to the revenue impact of some bill.
What the models do, and the politicians welcome, is sacrifice accuracy and honesty for certainty. Including, Merrill Matthews notes, a CBO estimate of the cost of Obamacare that was wildly wrong in the long run but politically useful for Democrats in the short run; of course they’ll be back for more. Or Katharine Hayhoe solemnly intones that “almost 40% more rain fell during Hurricane Harvey than would have otherwise” and it’s a rare brave soul who says “It is impossible to make such a calculation.”
Nowadays it’s very hard to beat a model except with another model. It’s very hard to convince people that when it comes to climate, nobody knows what’s going to happen including us. They end up sneering “You’re not a climate scientist” and going for a refill of snake oil. And there’s always some salesman waving a jug of the stuff. But this time we noticed. This time the models did enormous economic and social harm while getting the disease wrong.
If it turns out we did ourselves enormous harm by magical thinking about the humming boxes with the lights and the megaflops tended by egghead sages, what ever will the implications be? It depends in significant measure on whether those who habitually use models to silence critics are willing to admit that it’s not a sensible way to proceed.
The alternative is a bit unnerving. Curry’s paper began by quoting Voltaire: “Doubt is not a pleasant condition, but certainty is absurd.” So can we learn to live with doubt? Or must we persist in being absurd even when it hurts as badly as the excessively severe and protracted lockdowns have done and are continuing to do?