Skepticism about science and medicine

In search of disinterested science

Science is broken

Posted by Henry Bauer on 2017/11/21

Science is broken: Perverse incentives and the misuse of quantitative metrics have undermined the integrity of scientific research is the full title of an article published in the on-line journal AEON . I learned of it through a friend who was interested in part because the authors are at the university from which I retired some 17 years ago.

The article focuses on the demands on researchers to get grants and publish, and that their achievements are assessed quantitatively rather than qualitatively, through computerized scoring of such things as Journal Impact Factor and numbers of citations of an individual’s work.

I agree that those things are factors in what has gone wrong, but there are others as well.

The AEON piece is an abbreviated version of the full article in Environmental Engineering Science (34 [2017] 51-61; DOI: 10.1089/ees.2016.0223). I found it intriguing that the literature cited in it overlaps very little with the literature with which I’ve been familiar. That illustrates how over-specialized academe has become, and with that the intellectual life of society as a whole. There is no longer a “natural philosophy” that strives to integrate knowledge across the board, from all fields and specializations; and there are not the polymath public intellectuals who could guide society through the jungle of ultra-specialization. So it is possible, as in this case of “science is broken”, for different folk to reach essentially the same conclusion by extrapolating from quite different sets of sources and quite independently of one another.

I would add more factors, or perhaps context, to what Edwards and Roy emphasized:

The character of research activity has changed out of sight since the era or “modern science” began; for example, the number of wannabe “research universities” in the USA has tripled or quadrupled since WWII — see “Three stages of modern science”; “The science bubble”; chapter 1 in Science Is Not What You Think [McFarland 2017].

This historical context shows how the perverse incentives noted by Edwards and Roy came about. Honesty and integrity, dedication to truth-seeking above all, were notable aspects of scientific activity when research was something of an ivory-tower avocation; nowadays research is so integrated with government and industry that researchers face much the same difficulties as professionals who seek to practice honesty and integrity while working in the political realm or the financial realm: the system makes conflicts of interest, institutional as well as personal, inevitable. John Ziman (Prometheus Bound, Cambridge University Press) pointed out how the norms of scientific practice nowadays differ from those traditionally associated with science “in the good old days” (the “Mertonian” norms of communality, universality, disinterestedness, skepticism).

My special interest has long been in the role of unorthodoxies and minority views in the development of science. The mainstream, the scientific consensus, has always resisted drastic change (Barber, “Resistance by scientists to scientific discovery”, Science, 134 [1961] 596–602), but nowadays that resistance can amount to suppression; see “Science in the 21st century”; Dogmatism in Science and Medicine: How Dominant Theories Monopolize Research and Stifle the Search for Truth [McFarland, 2012]). Radical dissent from mainstream views is nowadays expressed openly almost only by long-tenured full professors or by retired people.

I’m in sympathy with the suggestions at the end of the formal Edwards and Roy paper, but I doubt that even those could really fix things since the problem is so thoroughgoingly systemic. Many institutions and people are vested in the status quo. Thus PhD programs will not change in the desired direction so long as the mentoring faculty are under pressure to produce more publications and grants, which leads to treating graduate students as cheap hired hands pushing the mentor’s research program instead of designing PhD research as optimum for neophytes to learn to do independent research. The drive for institutional prestige and status and rankings seems the same among university leaders, and they seek those not by excelling in “higher education” but by winning at football and basketball and by getting and spending lots of grant money on “research”. How to change that obsession with numbers: dollars for research, games won in sports?

That attitude is not unique to science or to academe. In society as a whole there has been increasing pressure to find “objective” criteria to avoid the biases inherent inevitably in human judgments. Society judges academe by numbers — of students, of research expenditures, of patents, of magnitude of endowment , etc. — and we compare nations by GDP rather than level of satisfaction among the citizens. In schools we create “objective” and preferably quantifiable criteria like “standards of learning” (SOLs), that supersede the judgments of the teachers who are in actual contact with actual students. Edwards and Roy cite Goodhart’s Law, which states that “when a measure becomes a target, it ceases to be a good measure”, which was new to me and which encapsulates so nicely much of what has gone wrong. For instance, in less competitive times, the award of a research grant tended to attest the quality of the applicant’s work; but as everything increased in size, and the amount of grants brought in became the criterion of quality of applicant and of institution, the aim of research became to get more grants rather than to do the most advancing work that would if successful bring real progress as well as more research funds. SOLs induced teachers to cheat by sharing answers with their students before giving the test. And so on and on. The cart before the horse. The letter of every law becomes the basis for action instead of the human judgment that could put into practice the spirit of the law.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s