See Comments down arrow

But... but... peer review!!!

05 Oct 2022 | News Roundup

In the ongoing alarmist misunderstanding of science as a club where unanimous dogmas are enforced against all questioning, the all-purpose shield against criticism is “it’s been peer reviewed”. Intelligent opinions backed by plausible evidence can be swept aside if not vetted by the nameless peers, whereas the stamp of approval from said reviewers is held up as a sign of infallibility that even the Popes of old would envy. So it’s a major story that London-based Hindawi, a subsidiary of the publisher John Wiley & Sons that publishes a broad range of scientific journals, is retracting over 500 papers in 16 journals because of “suspicious and unethical activities”. Likewise the Institute of Physics, which also publishes science journals, retracted 500 papers at the start of September for the same reason (after having retracted 350 papers in February). And PLoS, another scientific publisher, retracted 100 papers in August. All the papers in question slipped through the net in just the last two years. What else is out there?

In drawing attention to this issue we want to say that Hindawi and John Wiley & Sons and the others seem to be doing the right thing. Of course they should have exercised better oversight. But when they realized they had a problem, they acted decisively and spoke out. What is everyone else going to do?

The most recent scandal broke, the Epoch Times reports, when one of the journal editors noticed something odd about a paper and alerted Hindawi. And their VP of researcher and publishing services, Richard Bennett, “told the Retraction Watch blog that the review uncovered ‘coordinated peer review rings,’ which featured reviewers and editors coordinating to get papers through peer review.”

So far so good. We’re not as impressed that a spokesperson for John Wiley & Sons issued one of those greasy boilerplate statements, namely “All Hindawi journals employ a series of substantial integrity checks before articles are accepted for publication”, but if it were true the scandal would never have arisen in the first place.

Perhaps substantial does not mean what they think it means, especially since the spokesperson, “a senior vice president” at Wiley named Liz Ferguson, added that “Since identifying this unethical activity and breach of our processes, we began proactively adding further checks and improving our processes and continue to do so.” But the real flag here is that there’s such a thing as “Retraction Watch” at all.

It is an open secret that the whole process of peer review is not what it seems. While the public might assume that reviewers are independent of the authors and at the very least read the papers carefully for logical or factual errors and in some cases attempt to replicate the results, the fact is that it’s usually unpaid, cursory and far too often “pal review” in which Professor A praises Professor B’s paper this month and B praises A’s the next. We have commented on this point previously, including its connection with the insufficiently notorious “replication crisis” in which efforts to reproduce the findings in peer-reviewed papers astonishingly often fail, sometimes because of outright data fabrication by researchers either inherently dishonest or panicked that if they do not publish they will perish, another flaw in academia though one for another day. And we have also commented on its obvious implications including that if a journal devoted to climate science is in the hands of alarmist editors, it may very well employ alarmist reviewers to rubber-stamp alarmist researchers. (Who are themselves strangely willing to lash out at peer-reviewed research they don’t like, incidentally.)

If peer review were something other than pal review it would not be the case that, as Roger Pielke Jr. calculated a couple of years ago, nearly 17,000 peer-reviewed studies using RCP8.5 have been published. And as Judith Curry commented last year about an especially egregious paper by Michael Mann and three others, “Peer review is sooooo broken.”

Indeed. Ferguson also said that some of those involved with the retracted articles may have been “unwitting participants” in a scheme that involved “manipulation of the peer review process and the infrastructure that supports it.” But at a certain point we have to wonder if the problem isn’t manipulation of the process and the infrastructure, it’s the way they’re designed.

In a society in which trust in established institutions is rapidly declining, it is commendable that those involved are taking vigorous steps. And as we noted, Hindawi is not some sloppy outlier. Wiley has contacted other publishers and organizations, presumably not just to warn them of the problems they have found in their own work but to caution them to take a close look at their own.

Meanwhile, next time someone tries to invoke “peer review” to shut down debate, tell them honesty and common sense are what we really need, not dogmatic gatekeepers waving scrolls and mumbling incantations.

2 comments on “But... but... peer review!!!”

  1. The publish-or-perish attitude of academia will inevitably result in vast numbers of papers being published which have little or nothing to say, thus exponentially increasing the tempation to authors of simply make things up.
    The glory days of phyisics were arguably in the first half of the twentieth century when both quantum and relativity theories were developed. The annual output in that period of one of the most important physics journals, Physical Review, typically consisted of a single, moderately sized volume. Today there are 17 separate Phys Rev journals and the annual output of each, if printed, might well occupy several fat volumes. Does anyone seriously believe that every paper published in them has someting important to say?

Leave a Reply

Your email address will not be published. Required fields are marked *