One strange feature of climate science is that it is settled. Unlike every other field in which new discoveries are not just made but welcomed and indeed sought after by the diligent and the ambitious. For instance this new planet that “dashes” the view that stars more than three times the mass of Sol “emit so much radiation that they were thought to torch the planetary formation process”. Another strange thing is that climate science is usually declared settled by people unfamiliar with it, something also resisted in other fields. And by settled they mean that it’s only ever worse than the infallible experts thought. Also strange is that peer review is believed to be a silver bullet that slays all skeptics. Which is why the dramatic fall from grace of rising star Jonathan Pruitt, spider scholar and Canada 150 Chair at McMaster University, whose peer-reviewed papers are now believed to have been based on phony data, has serious implications for global warming even if you don’t believe those stories that “NATURE’S CREEPIEST CRAWLER IS THRIVING AS CLIMATE CHANGE WARMS THE ARCTIC”.

We take no pleasure in the ruin of a career. Which seems to be occurring with Pruitt, now suspended from his academic chair. Not only have over a dozen of his papers been retracted, so has his doctoral dissertation. Perhaps he succumbed to the pressure to publish or perish, or the temptations of early fame and success, and perhaps both require attention. Whatever happened, we hope he can find redemption. But this article isn’t about him. It’s about this notion that peer review draws a clear line between solid work and eccentric rubbish or venal fakery.

In fact the “replication crisis“ in scholarly publishing has been an issue for nearly two decades now, since the publication of John P.A. Ioannidis’ provocative 2005 paper “Why Most Published Research Findings Are False”. Most. Scary, huh? Well, yes, especially as this essay provoked dedicated and honourable people in field after field to start exploring what was being published in their area and, even more painfully, how. Indeed it is a tribute to the integrity as well as the intelligence of researchers worldwide that this Ioannidis’ paper is “the most-accessed article in the history of Public Library of Science as of 2020, with more than 3 million views.”

There’s a lesson there about how science is done. It’s not by enforcing orthodoxy, it’s by relentlessly looking for errors. But also about how it has gone wrong across the board.

A significant problem is the pressure on researchers to discover and journals to public something remarkable. A paper saying pretty much what we already thought is not going to get you tenure, nor will one that says gee, nothing very clear emerges from the fog. In other fields, especially medicine-related, there may also be pressure to prove something works. And all the usual human problems emerge as well, from ego to ambition to congeniality. No, really; one problem with “peer review” is that it can become “pal review” because people want to be nice to their colleagues as well as because they want to get good reviews in return.

To say so is not to denounce science. Or even peer review in principle. Ioannidis himself has said, “Science is the best thing that has happened to human beings… but we can do it better.” On reading this passage we wondered if he’s married; it turns out he is so don’t tell his wife or kid what he said. But while we think there are other matters to celebrate in life, we agree that the appropriate response to problems with peer review is to do it better, not chuck the concept. Pay more. Hide the names of authors. Require some replication of results. Reward dullness of findings as well as of prose. That sort of thing.

In climate science, where there is a frightening degree of zealotry, the closed nature of peer review is especially far from being a guarantee of quality. On the contrary, it gives alarmists a strong incentive to take over the process and exclude challenges to orthodoxy. Which in turn allows statistically flawed studies to claim a non-existent consensus based in part on who is most often admitted to the charmed inner circle of peer-reviewed publications. If these people were economists, they would instinctively recognize the perverse incentives created there, as when companies with products to sell are allowed to fund studies of their safety or effectiveness. (Or politicians are allowed to silence critics.)

Human beings are not essentially villains. But we should not ask more sustained virtue of them than can reasonably be expected over a long period of temptation.

Including the temptation to say anyone who isn’t “peer reviewed” is just a yokel when it comes to global warming. If you make it a good way to avoid the hard work of constructing arguments and examining one’s own position, people will resort to it with increasing frequency, first perhaps out of haste, then habit and finally to avoid a humiliating retreat.. Such an outcome won’t help anybody in the end.