[CDN Note: The CO2Science archive is extremely deep and sometimes we present older studies that remain timely and relevant, given that ideas that stand the test of time are the core of the scientific method.]
From CO2Science: In the title of their newly published article in the journal Scientometrics, Jankó et al. (2017) ask the important question “Is climate change controversy good for science?” Their answer, which we will divulge later, came about via a somewhat unique analysis, which compared the reference lists of two major reports published by the Intergovernmental Panel on Climate Change (IPCC; AR4 and AR5) with that published by the Nongovernmental International Panel on Climate Change (NIPCC; Climate Change Reconsidered (CCR) and Climate Change Reconsidered II (CCRII)). More specifically, the comparison involved (1) entering all references from the four reports into a database, (2) sorting them by author and by journal, (3) comparing the overlap and differences in citations between the two publishing entities and (4) examining the evolution (i.e., similarities and differences) of citations by each entity between their two reports. And what did these several actions reveal?
Paper cited: Jankó, F., Vancsó, J.P. and Móricz, N. 2017. Is climate change controversy good for science? IPCC and contrarian reports in the light of bibliometrics. Scientometrics 112: 1745-1759.
Not surprisingly, there were some similarities and differences among the references cited in the IPCC and NIPCC reports. In terms of differences, the IPCC tended to favor citations from journals that are focused more on the modelling enterprise, whereas the NIPCC tended to favor publications in the paleo-sciences. By way of similarity, when comparing the AR4 and CCR reports, both organizations derived references from an overlapping set of 126 journals, which number increased to 198 for the ensuing AR5 and CCRII tomes. However, Jankó et al. report that “the sceptics have broadened their spectrum of journals more than the IPCC,” as the NIPCC added 170 new journals to their citation list between their two reports, while the IPCC added only 158.
Another interesting finding is seen in their examination of who each organization was citing. In-text analysis of the IPCC’s AR5 report revealed that 19 out of the 20 most frequently cited authors in that report were directly involved in the compilation of it. And though the remaining person, J. Hansen, was not officially involved in producing AR5, he participated in the production of at least one prior IPCC report (Third Assessment) as a Contributing Author. Similar analysis of the AR4 report revealed that 14 out of the 16 most frequently cited IPCC authors were involved with the writing of that report. Yet, here again, the remaining two individuals were directly involved in the production of the IPCC’s preceding Third Assessment Report. Such findings indicate the IPCC report authors are most intent on citing their own work, thereby promoting their own interests and findings above the work of others. In contrast, only four of the 18 most frequently cited authors of CCRII, and only one of the 13 most frequently cited authors of CCR, were involved in the compilation of those reports. Thus, the NIPCC reports present a greater degree of independence among its authors and the material they produce and cite than that of the IPCC.
Finally, returning to Jankó et al.’s question posed in the title of their paper -- “Is climate change controversy good for science?” – in summing up their analysis they write that “the competitive situation created by the publications of the NIPCC reports ... is beneficial for climate science in general; it fosters knowledge creation, i.e. the reviewing process, mobilizing a growing number of references into review.” And while this knowledge creation is important, Jankó et al. caution that “without an explicit dialogue between the [NIPCC and IPCC] reports, there is no chance to mitigate climate change controversy itself.” In other words, (1) there is no scientific consensus, (2) the debate is not settled, (3) nor will it ever go away until the closed-minded circular group-think of the IPCC authors properly recognize and address the counter-theories presented by the NIPCC in its reports. Good luck getting that to happen!