Something old, something new

Several of my books had gone out of print over the years, including my personal favorites: The Enigma of Loch Ness and To Rise Above Principle: The Memoirs of an Unreconstructed Dean. Now they have been re-issued by Wipf & Stock, with additional material in the Dean’s Memoirs.

The Enigma of Loch Ness is not an argument for or against the reality of Nessies. As the sub-title, Making Sense of a Mystery, suggests, the book addresses such questions as
“Why has science not taken an interest in this claim?”
“What’s different about looking for Nessies and doing science, say, searching for new species?”
In discussing what differentiates science from not-science, I introduce the concept that scientific activity has 3 aspects: facts, methods, and theories. Science progresses normally by not rocking the boat in any of those aspects. Occasional scientific revolutions see dramatic change in one of those aspects. When two are in question, the claim is likely to be ignored by the mainstream for quite a long time (Mendelian genetics, continental drift). Claims that are unorthodox in all 3 aspects tend to be shunned as pseudo-science.

To Rise Above Principle was published under a pen-name because I did not want my semi-imaginative scenarios and semi-invented characters to be identified with my institution or its people. A year later, when that danger no longer seemed pressing, I had the opportunity to reveal my identity at an annual meeting of the Council of Colleges of Arts & Sciences, and this new edition of the book contains the text of what I said on that occasion. Another appendix has my introduction to a panel concerned with what a dean’s career involves, and a third appendix reproduces my change of mind on the matter of homosexuality, earlier published in a book review and posted on my website.

Both books received high critical praise when they were published. All but one review of the Nessie book noted that it was not an argument for or against and praised my evenhandedness despite my confessed personal view. The demurring review misled by implying that the book was an attempt to argue for the reality of Nessies; it had been written by someone who had earlier asked without success to be invited to give a seminar at my university — an illustration of the pitfalls of “peer” review [Peer review and consensus (Scientific literacy, lesson 2)].

🙂     I recommend both books wholeheartedly and unashamedly     🙂

Peer review and consensus (Scientific literacy, lesson 2)

The conventional wisdom persistently points to peer review in science and medicine as the safeguard of quality and reliability. But peer review reflects the characteristics of the peer reviewers, and if the latter are biased or lazy or incompetent, then so will be the peer review, and neither quality nor reliability are safeguarded (Scientific Literacy and the Myth of the Scientific Method, University of Illinois Press, 1992).

Consider how trustworthy — or not — peer review would be in other human activities, say in politics. Imagine it was decided that the remuneration of Congressional representatives would depend on how well they had furthered the best interests of the nation, and that the judgment were made via peer review awarding grades of A through E. Not uncommonly, Republican peer reviewers would grade Democrats rather lower than Democrat peer reviewers would, and vice versa, wouldn’t they? Not in every individual case, of course, but statistically, on average.

Quite generally, when peer reviewers are competitors or colleagues of those being reviewed, one should not expect purely disinterested evidence-based judgments to be made.

Scientists are human and subject to all the psychological and social weaknesses to which humans are prone (Scientific literacy in one easy lesson). It displays scientific illiteracy to imagine that peer review in science or medicine guarantees the integrity, objectivity, and quality of funding and hiring and publication decisions.

The reliability or otherwise of science itself runs rather in parallel with the reliability or otherwise of peer review. In the first era of modern science, conflicts of interest were largely personal (From Dawn to Decadence: The Three Ages of Modern Science), and the peer review exerted by the scientific community as a whole could average out biases and individual animosities and friendships — all the more so because the peer review was quite informal and amounted largely to testing and discussing claims already made public.

In the second era, careerism could affect peer review and the progress of science, but the stakes were not very high, and again the overall judgment of the scientific community was relatively disinterested, averaging out personal biases. It was still relatively informal.

In the present era, however, judgments made by peer review emerge from a highly formalized and bureaucratic system and not from the sum of individual opinions from essentially all the pertinent members of the scientific community, which had been the case in earlier times when review was largely post-publication, whereas now it is pre-publication. Furthermore, today’s competition is cutthroat and the stakes can be very high. For these reasons, contemporary peer review is not a sound way of judging what is the best science or the best scientists. Just as fraud in science has become a major preoccupation in the last few decades, so have issues about the reliability of peer review, and for the same reasons.

One dramatic episode saw the effective demise of the journal Medical Hypotheses. The overt reason given for emasculating the journal was that it had functioned under editorial rather than peer review, but the actual reason was that the journal had published well documented articles debunking a mainstream dogma (Chapter 3, “A Public Act of Censorship: Elsevier and Medical Hypotheses”, in Dogmatism in Science and Medicine: How Dominant Theories Monopolize Research and Stifle the Search for Truth).

Journal editors are in prime position to evaluate the pros and cons of peer review. It takes little time as editor to learn that for any given submitted manuscript, one can obtain favorable reviews or unfavorable reviews by appropriate choice of peer reviewers. The integrity of science depends on the integrity of editors and others who are forced by the prevailing system to resort to peer review to justify what they do. But there is nothing automatically sound about peer review:

“Classical peer review: an empty gun”, says Richard Smith, former editor of the British Medical Journal and chief executive of the BMJ Publishing Group, now on the board of the Public Library of Science and editor of Cases Journal, which uses only a minimal peer-review system. Smith cites Drummond Rennie, deputy editor of the Journal of the American Medical Association (JAMA) and “intellectual father of the international congresses of peer review that have been held every four years since 1989”:

“If peer review was a drug
it would never be allowed onto the market”.
“Peer review would not get onto the market
because we have no convincing evidence of its benefits
 but a lot of evidence of its flaws. . . .
[P]eer review . . . is . . .
an ineffective, slow, expensive, biased, inefficient,
 anti-innovatory, and easily abused lottery:
 the important is just as likely to be filtered out
 as the unimportant”
—— Breast Cancer Research, 12 [suppl. 4, 2010] S13

Richard Horton, editor of The Lancet, has pointed out that

“Peer review … is simply a way
 to collect opinions from experts in the field.
Peer review tells us about the acceptability,
 not the credibility, of a new finding”
—— Health Wars: On the Global Front Lines of Modern Medicine,
New York Review Books, 2003, p. 306

Peer review cannot even be relied on to detect purely technical errors, as Richard Smith (above) found by deliberately inserting errors into manuscripts before they were reviewed. The sorry incompetence of statistical analysis in the medical scientific literature has been exposed innumerable times over the course of two or three decades with no sign of improvement; see for example D. G. Altman, “The scandal of poor medical research”, British Medical Journal, 308 (1994) 283, and nearly a decade later, “Poor-quality medical research: what can journals do?”, JAMA, 287 (2002) 2765-2767; also John P. A. Ioannidis, “Why most published research findings are false”, PLoS Medicine, 2 (2005) e124, and John P. A. Ioannidis & Orestis A. Panagiotou, “Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses”, JAMA, 305 (2011) 2200-10.

Some of the flaws of peer reviewing were specifically criticized by stem-cell researchers (“Peer review trickery?”, Jef Akst. The Scientist [2nd February 2010]). [I suggest that it is likely no coincidence that stem-cell research is presently a particularly “with-it”, faddish, competitive field].
The editor of Nature, Philip Campbell, disagreed by outlining how carefully he and his editorial team try to guard against bias (“Peering into review”, Nature Medicine, 16 [#3, 2010] 239). But Campbell’s response comes from a thoroughgoingly Establishment source, debilitated by cognitive dissonance from recognizing its own fallibility; the editorial is worth reading carefully as an exemplar of self-serving generalities and rhetorical obfuscation. Nature’s inability to see itself as others see it has been illustrated amply on other occasions, for example, when it insisted in 1997 that there was no reason to require authors to reveal financial conflicts of interest, and subsequently changed that policy only reluctantly, in 2001, revealing by its commentary that Nature still did not understand that conflicts of interest inevitably tend to induce bias (p. 159, Dogmatism in Science and Medicine).  An inevitable tendency produces statistically significant effects, no matter that the tendency might have no effect in any given instance. That Nature fails to grasp that straightforward fact is surely the consequence of cognitive dissonance and not stupidity. I suggest that Nature’s universally acknowledged role as one of the two (with Science) most authoritative publications in all of science renders it incapable of appreciating the fundamental flaws that characterize the modern age of science (p. 67 in Dogmatism in Science and Medicine; From Dawn to Decadence: The Three Ages of Modern Science).

Innumerable studies have recognized that peer review discriminates against originality and novelty. There are at least three quite classic sources:
Bernard Barber, “Resistance by scientists to scientific discovery”, Science, 134 (1961) 596-602.
Gunther Stent, “Prematurity and uniqueness in scientific discovery”, Scientific American, December 1972, 84-93.
Ernest B. Hook, (ed)., Prematurity in Scientific Discovery: On Resistance and Neglect, University of California Press, 2002.

Despite all this, peer reviewing remains the routine practice almost everywhere. One reason may be that no acceptable alternative seems in sight. Another is that the present system is satisfactory to established institutions, for it safeguards their established status. Furthermore peer review enables decision-makers to avoid taking personal responsibility for their decisions, as when editors make their choices of reviewers. A less obvious reason is that those who maintain, use, nurture, and defend the system are largely scientifically illiterate. Typically they came from within the ranks of practicing scientists and have not become acquainted with the insights acquired over decades by scholars of STS.

*                    *                    *                    *                    *                    *                    *                    *

“Consensus” is commonly cited as certifying the reliability of pronouncements about science just as is peer review. That is unsound for the same reasons. The mainstream view, the “scientific consensus”, is established through processes governed by peer review. When peer review is flawed, so is the consensus. One can hardly speak more truly than Michael Crichton on this:

“I regard consensus science
 as an extremely pernicious development
that ought to be stopped cold in its tracks.
Historically, the claim of consensus has been
 the first refuge of scoundrels;
 it is a way to avoid debate
 by claiming that the matter is already settled.
Whenever you hear the consensus of scientists
 agrees on something or other,
 reach for your wallet, because you’re being had….
 Consensus is invoked only in situations
 where the science is not solid enough.
Nobody says the consensus of scientists agrees that E=mc 2 .
Nobody says the consensus is that the sun is 93 million miles away.
It would never occur to anyone to speak that way”
— “Aliens cause global warming”,
  Caltech Michelin Lecture, 17 January 2003.