Peer Review

One factor that clearly distinguishes informed consumers of science and the general public is the attitude these groups have towards the process of peer-review. The general public entertains unrealistic, highly-idealised visions of a process by which scientific research is assessed by peers. In theory, peer review is supposed to act as a filter, weeding out the crackpots; in practice, it often turns out to be a way to enforce orthodoxy.

Copernicus’s heliocentric cosmology, Galileo’s mechanics, Newton’s gravity and equations of motion – these ideas never appeared in journal articles. They appeared in books that were reviewed, if at all, by associates of the author. The peer-review process as we know it was instituted after the Second World War, largely due to the huge growth of the scientific enterprise and the enormous pressure on academics to publish ever more papers.

Since the 1950s, peer-review has worked as follows: a scientist wishing to publish a paper in a journal submits a copy of the paper to the editor of a journal. The editor forwards the paper to several academics whom he considers to be experts on the matter, asking whether the paper is worthy of publication. These experts – who usually remain anonymous – submit comments about the paper that constitute the “peer review”. The referees judge the content of the paper on criteria such as the validity of the claims made in the paper, the originality of the work, and whether the work, even if correct and original, is important enough to be worthy of publication. Often, the journal editor will require the author to amend his paper in accordance with the recommendations of the referees.

Prior to the War, university professors were mainly teachers, carrying a teaching load of five or six courses per semester (a typical course load nowadays is one or two courses). Professors with this onerous teaching burden were not expected to write papers. The famous philosopher of science Sir Karl Popper wrote in his autobiography that the dean of the New Zealand university where Popper taught during World War II said that he regarded Popper’s production of articles and books a theft of time from the university.

But at some point, universities came to realise that their prestige – and with it the grants they received from governments and corporations – depended not so much on the teaching skills of their professors but rather on the scholarly reputation of these professors. And this reputation could only be enhanced through publications. Teaching loads were reduced to allow professors more time for research and the production of papers; salaries began to depend on one’s publication record. Before the War, salaries of professors of the same rank (associate professor, assistant professor, adjunct professor, full professor etc.) were the same (except for an age differential, which reflected experience). Nowadays, salaries of professors in the same department of the same age and rank can differ by more than a factor of two.

One consequence of all this is that the production of papers has increased by a factor of more than one thousand over the past fifty years. The price paid for this fecundity is a precipitous decline in quality. Before the War, when there was no financial incentive to publish papers, scientists wrote them as a labour of love. These days, papers are written mostly to further one’s career. One thus finds that nowadays, most papers are never cited by anyone except their author(s).

Philip Anderson, who won a Nobel Prize for physics, writes that

In the early part of the postwar period [a scientist’s] career was science-driven, motivated mostly by absorption with the great enterprise of discovery, and by genuine curiosity as to how nature operates. By the last decade of the century far too many, especially of the young people, were seeing science as a competitive interpersonal game, in which the winner was not the one who was objectively right as [to] the nature of scientific reality, but the one who was successful at getting grants, publishing in Physical Review Letters, and being noticed in the news pages of Nature, Science, or Physics Today… [A] general deterioration in quality, which came primarily from excessive specialization and careerist sociology, meant quite literally that more was worse.[1]

More is worse. As Nature puts it, “With more than a million papers per year and rising, nobody has time to read every paper in any but the narrowest fields, so some selection is essential. Authors naturally want visibility for their own work, but time spent reading their papers will be time taken away from reading someone else’s.” The number of physicists has increased by a factor of one thousand since the year 1900. Back then, ten percent of all physicists in the world had either won a Nobel Prize or had been nominated for it. Things are much the same in chemistry. The American Chemical Society made a list of the most significant advances in chemistry over the last 100 years. There has been no change in the rate at which breakthroughs in chemistry have been made in spite of the thousand-fold increase in the number of chemists. In the 1960s, US citizens were awarded about 50 000 patents in chemistry-related areas per year. By the 1980s, the number had dropped to 40 000. But the number of papers has exploded. One result of this publish-or-perish mentality is that groundbreaking papers are often rejected because they are submitted to referees who are incapable or unwilling to recognise novel ideas. Consider these examples.

Rosalyn Yalow won the Nobel Prize in Physiology in 1977. She describes how her Nobel-winning paper was received: “In 1955 we submitted the paper to Science… the paper was held there for eight months before it was reviewed. It was finally rejected. We submitted it to the Journal of Clinical Investigations, which also rejected it.”[2]

Günter Blobel also won a Nobel Prize in Physiology, in 1999. In a news conference given just after he was awarded the prize, Blobel said that the main problem one encounters in one’s research is “when your grants and papers are rejected because some stupid reviewer rejected them for dogmatic adherence to old ideas.” According to the New York Times, these comments “drew thunderous applause from the hundreds of sympathetic colleagues and younger scientists in the auditorium.”[3]

Mitchell J. Feigenbaum thus described the reception that his revolutionary papers on chaos theory received: “Both papers were rejected, the first after a half-year delay. By then, in 1977, over a thousand copies of the first preprint had been shipped. This has been my full experience. Papers on established subjects are immediately accepted. Every novel paper of mine, without exception, has been rejected by the refereeing process. The reader can easily gather that I regard this entire process as a false guardian and wastefully dishonest.”[4]

Theodore Maiman invented the laser, an achievement whose importance is not doubted by anyone. The leading American physics journal, Physical Review Letters, rejected Maiman’s paper on constructing a laser.[5]

John Bardeen, the only person to have ever won two Nobel Prizes in physics, had difficulty publishing a theory in low-temperature solid-state physics that went against the paradigm.[6]

Stephen Hawking needs no introduction. According to his first wife Jane, when Hawking submitted to Nature what is generally regarded as his most important paper on black hole evaporation, the paper was initially rejected.[7] The physicist Frank J. Tipler writes that “I have heard from colleagues who must remain nameless that when Hawking submitted to Physical Review what I personally regard as his most important paper, his paper showing that a most fundamental law of physics called ‘unitarity’ would be violated in black hole evaporation, it, too, was initially rejected.”

 Conventional wisdom in contemporary geophysics holds that the Hawaiian Islands were formed sequentially as the Pacific Plate moved over a hot spot deep inside the Earth. This idea was first developed in a paper by the Princeton geophysicist Tuzo Wilson. Wilson writes: “I… sent [my paper] to the Journal of Geophysical Research. They turned it down… They said my paper had no mathematics in it, no new data, and that it didn’t agree with the current views. Therefore, it must be no good. Apparently, whether one gets turned down or not depends largely on the reviewer. The editors, too, if they don’t see it your way, or if they think it’s something unusual, may turn it down. Well, this annoyed me…”[8]

There is not much incentive for referees to carefully adjudicate their fellow-scientists’ papers. As Nature puts it: “How much time do referees expend on peer review? Although referees may derive benefits from reviewing, it still represents time taken away from other activities (research, teaching and so forth) that they would have otherwise prioritized. Referees are normally unpaid but presumably their time has some monetary value, as reflected in their salaries.”

In 2006, Nature published an essay by Charles G. Jennings, a former editor with the Nature journals and former executive director of the Harvard Stem Cell Institute. As an editor, Jennings was intimately familiar with the peer-review system, and knows full well how badly misunderstood this process is by the public:

Whether there is any such thing as a paper so bad that it cannot be published in any peer reviewed journal is debatable. Nevertheless, scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.

Jennings writes that “many papers are never cited (and one suspects seldom read)”. These papers are written, to a large extent, because “To succeed in science, one must climb this pyramid [of journals]: in academia at least, publication in the more prestigious journals is the key to professional advancement.” Advancement, in this context, is measured by career rewards such as recruitment and promotion, grant funding, invitations to speak at conferences, establishment of collaborations and media coverage.

***

Many in the scientific community recognise the ills that plague the peer-review process, and experiments are being conducted to improve – or sidestep – the current dispensation. For example, some journals no longer grant referees the protection of anonymity. Instead, reviewers are identified and their critiques of papers are made available to the author of the paper being reviewed. The author is then able to defend his paper. This may ameliorate the problem of reviewers who hamper the publication of a paper for less than noble reasons (such as professional jealousy).

At any rate, informed consumers of science understand that peer-review is far from perfect. It is an efficient way of strangling new ideas, rather than a vehicle for promoting truly novel ideas. The peer-review system often stifles true innovation, allowing the reigning paradigm to squash all competition unfairly. This is especially true in controversial areas like biological evolution.

***

References:

My two main references for this post are:

  1. An essay by the physicist Frank J. Tipler entitled Refereed Journals: do they insure quality or enforce orthodoxy? The essay appeared in the volume Uncommon Dissent: Intellectuals who find Darwinism Unconvincing, William A. Dembski (editor), ISI Books, 2004.
  2. A 2006 editorial in Nature, available here: http://www.nature.com/nature/peerreview/debate/nature05032.html. Retrieved 26th May 2013.

[1] Philip Anderson, in Brown, Pais and Pippard, editors, Twentieth Century Physics, American Institute of Physics Press, 1995,page 2029.

[2] Walter Shropshire Jr., editor, The Joys of Research, Smithsonian Institution Press, 1981, page 109.

[3] New York Times, 12th October 1999, page A29.

[4] Mitchell J. Feigenbaum, in Brown, Pais and Pippard, editors, Twentieth Century Physics, American Institute of Physics Press, 1995, page 1850.

[5] Ibid. page 1426.

[6] Lillian Hoddeson, True Genius: The Life and Science of John Bardeen, Joseph Henry Press, 2002, page 300.

[7] Jane Hawking, Music to Move the Stars: A Life with Stephen Hawking, Trans-Atlantic Publications, 1999, page 239.

[8] Walter Shropshire Jr., editor, The Joys of Research, Smithsonian Institution Press, 1981, page 130.

Advertisements

One Response to “Peer Review”

  1. Menachem Mevashir Says:

    Comment submitted to David Berlinski’s daughter, Claire:

    http://berlinski.com/aboutclaire
    Dear Claire,

    I wonder if you could convey to your father how much I appreciated his comments about the flaws of Darwinist macro-evolution in the film Expelled.

    I also would like to ask if you could pose the following question to him:

    Your dad claims that Hitler’s Mein Kampf is permeated with Darwinist terminology, particularly if one can read the original German. But I also have read that the Nazis put Darwin’s writings on their list of banned books. Can he explain this apparent discrepancy?

    See point 6 here for example:

    http://www.library.arizona.edu/exhibits/burnedbooks/documents.htm

    6. Schriften weltanschaulichen und lebenskundlichen Charakters, deren Inhalt die falsche naturwissenschaftliche Aufkl�rung eines primitiven Darwinismus und Monismus ist (H�ckel). 6. Writings of a philosophical and social nature whose content deals with the false scientific enlightenment of primitive Darwinism and Monism (H�ckel).

    Did the Nazis blacklist Darwin simply to appease the Christian supporters of the Third Reich? Or did this reflect perhaps the genuine underlying Christian sentiments of its leaders?

    And finally how would he respond to the comment of Professor Jerry Coyne, evolutionary biologist from the University of Chicago, who when I asked him about the film Expelled, commented that Hitler did not build on Darwinism per se but rather on a radical form of selective breeding that has nothing to do with Darwinism.

    Thanks so much for your kind assistance,
    Sincerely,
    Michael Korn

    PS This is a very interesting blog by an Orthodox rabbi and professionally trained geologist in South Africa who often cites your father: Torah Explorer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: