A reader has kindly drawn my attention to an article by a physician, Jalees Rehman, which treads territory that will be familiar to readers of TorahExplorer. In this post, I reproduce some of Dr. Rehman’s points, interspersed with my comments.
Dr. Rehman begins by discussing what he terms the doctor mystique – “Doctors had previously been seen as infallible saviors who devoted all their time to heroically saving lives and whose actions did not need to be questioned” – a notion now rapidly crumbling. Informed patients have access to an immense amount of information with which to question the decisions of their physicians – “Instead of blindly following doctors’ orders, they want to engage their doctor in a discussion and become an integral part of the decision-making process.” In addition, patients nowadays are more aware of various factors that can skew doctors’ judgement:
The recognition that gifts, free dinners and honoraria paid by pharmaceutical companies strongly influence what medications doctors prescribe has led to the establishment of important new rules at universities and academic journals to curb this influence…
I discussed related issues in posts such as Dr. John Ioannidis and the Reality of Research and Dr. Ben Goldacre and the Reproducibility of Research.
Dr. Rehman’s essay, however, is devoted to another myth, one that he calls The Science Mystique. He correctly notes that it still persists where similar notions – the feminine mystique and the doctor mystique – have disappeared or are disintegrating. But Dr. Rehman is clear that the science mystique is vulnerable:
As with other mystiques, it [i.e. The Science Mystique] consists of a collage of falsely idealized and idolized notions of what science constitutes. This mystique has many different manifestations, such as the firm belief that reported scientific findings are absolutely true beyond any doubt, scientific results obtained today are likely to remain true for all eternity and scientific research will be able to definitively solve all the major problems facing humankind.
Quite right. Readers of Genesis and Genes will be familiar with a comment made by the physicist and philosopher Sir John Polkinghorne:
Many people have in their minds a picture of how science proceeds which is altogether too simple. This misleading caricature portrays scientific discovery as resulting from the confrontation of clear and inescapable theoretical predictions by the results of unambiguous and decisive experiments… In actual fact… the reality is more complex and more interesting than that.
Science is a human – read fallible – endeavour. Informed consumers of science understand that a host of factors influence research. Beyond the technical aspects of research, there are societal factors, political factors, ideological factors, financial factors and dozens more, some of which I discussed in the first chapter of Genesis and Genes. One consequence of this is that scientific findings come in a spectrum of credibility, ranging from solid to hopelessly speculative and ideological.
This science mystique is often paired with an over-simplified and reductionist view of science. Some popular science books, press releases or newspaper articles refer to scientists having discovered the single gene or the molecule that is responsible for highly complex phenomena, such as a disease like cancer or philosophical constructs such as morality.
Indeed. Most members of the public are not informed consumers of science, and are easily swayed by simplistic or exaggerated claims. A common example of exaggerated claims swallowed by the public comes from palaeontology. A fossil is unearthed and proclaimed as the latest earliest ancestor of human beings. After the media frenzy subsides and the public’s attention is diverted, the claims inevitably prove to be hollow. [For several excellent examples of the genre, see the chapter entitled Human Origins and the Fossil Record in Science and Human Origins.] This is true with respect to complicated concepts and phenomena like cancer or morality, as Dr. Rehman writes, but it is all the more true with respect to over-arching theories that purport to explain ultimate questions about the universe or life. The gullible public is unaware of the tremendous superstructure of assumptions, ideological commitments and technical difficulties that go into scientists’ absolutist statements about such subjects.
Dr. Rehman continues:
As flattering as it may be, few scientists see science as encapsulating perfection. Even though I am a physician, most of my time is devoted to working as a cell biologist. My laboratory currently studies the biology of stem cells and the role of mitochondrial metabolism in stem cells. In the rather antiquated division of science into “hard” and “soft” sciences, where physics is considered a “hard” science and psychology or sociology are considered “soft” sciences, my field of work would be considered a middle-of-the-road, “firm” science. As cell biologists, we are able to conduct well-defined experiments, falsify hypotheses and directly test cause-effect relationships. Nevertheless, my experience with scientific results is that they are far from perfect and most good scientific work usually raises more questions than it provides answers. We scientists are motivated by our passion for exploration, and we know that even when we are able to successfully obtain definitive results, these findings usually point out even greater deficiencies and uncertainties in our knowledge.
An important qualification is needed here. Researchers like Dr. Rehman are usually aware that in their field, perfection is elusive. But they are often largely ignorant of other fields, and may harbour unrealistic views of the reliability of research in those fields.
Readers of Genesis and Genes will recall chapter 3, in which I described how scientists from half-a-dozen different disciplines were attempting to determine the age of the Earth in the latter part of the 19th century. It was frequently the case that practitioners of one discipline, aware of the limitations of their own field, failed to understand that other fields were just as vulnerable, but for different reasons. This led to a situation in which a mirage was created that there was independent confirmation, arising from several different disciplines, regarding the age of the Earth. This turned out to be completely illusory.
Dr. Rehman now turns to reproducibility of research:
One key problem of science is the issue of reproducibility. Psychology is currently undergoing a soul-searching process because many questions have been raised about why published scientific findings have such poor reproducibility when other psychologists perform the same experiments. One might attribute this to the “soft” nature of psychology, because it deals with variables such as emotions that are difficult to quantify and with heterogeneous humans as their test subjects. Nevertheless, in my work as a cell biologist, I have encountered very similar problems regarding reproducibility of published scientific findings. My experience in recent years is that roughly only half the published findings in stem cell biology can be reproduced when we conduct experiments according to the scientific methods and protocols of the published paper.
Recall that earlier, Dr. Rehman characterised his field, cell biology, as a ‘firm’ science, somewhere between physics and psychology on a spectrum similar to the ‘proof continuum’ I discussed in Genesis and Genes. As he says, cell biology is an area of science where ostensibly objective parameters exist that should ensure the reproducibility of research. Alas, to a significant degree, reproducibility is elusive. Cell biology is not sociology or anthropology; nor are we talking about drug trials here (where as much as 90% of published studies may be wrong). Nonetheless, upwards of 50% of the research in cell biology is not reproducible. One is reminded of this passage in Genesis and Genes:
[Glenn] Begley [who served, for a decade, as head of global cancer research at Amgen] met for breakfast at a cancer conference with the lead scientist of one of the problematic studies. “We went through the paper line by line, figure by figure,” said Begley. “I explained that we re-did their experiment 50 times and never got their result. He said they’d done it six times and got this result once, but put it in the paper because it made the best story. It’s very disillusioning.”
On the other hand, we devote a limited amount of time and resources to replicating results, because there is no funding available for replication experiments. It is possible that if we devoted enough time and resources to replicate a published study, tinkering with the different methods, trying out different batches of stem cells and reagents, we might have a higher likelihood of being able to replicate the results. Since negative studies are difficult to publish, these failed attempts at replication are buried and the published papers that cannot be replicated are rarely retracted. When scientists meet at conferences, they often informally share their respective experiences with attempts to replicate research findings. These casual exchanges can be very helpful, because they help us ensure that we do not waste resources to build new scientific work on the shaky foundations of scientific papers that cannot be replicated.
The difficulty of publishing negative results and the lack of incentive to verify other researchers’ results are recognised as major contributors to systemic problems within contemporary science. The average member of the public labours under the illusion that mechanisms such as peer-review suffice to ensure that whatever is published in a mainstream journal is infallible. This, of course, constitutes child-like naivety. As Nature put it in a 2006 editorial, “Scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.”
Most scientists are currently struggling to keep up with the new scientific knowledge in their own field, let alone put it in context with the existing literature. As I have previously pointed out, more than 30-40 scientific papers are published on average on any given day in the field of stem cell biology. This overwhelming wealth of scientific information inevitably leads to a short half-life of scientific knowledge… What is considered a scientific fact today may be obsolete within five years.
Quite true. As I wrote in Genesis and Genes,
A paper published in the Proceedings of the National Academy of Scientists in 2006 noted that “More than 5 million biomedical research and review articles have been published in the last 10 years.” That’s an average of 1370 papers per day. And this is just biomedical research.
This deluge of information, and the fact that “What is considered a scientific fact today may be obsolete within five years”, has important repercussions for informed consumers of science. Those who follow the evolution debate are aware of how the ephemeral nature of scientific knowledge can have an impact on what was only recently considered absolute. Whether it is Tree of Life research, Junk DNA or the discovery of numerous instances of Lamarckian heredity, there have been breathtaking turnarounds in recent years. Basic prudence dictates that when evolutionary biologists invoke ‘overwhelming evidence’ on the basis of whatever, that their claims be taken with a sack of salt.
One aspect of science that receives comparatively little attention in popular science discussions is the human factor. Scientific experiments are conducted by scientists who have human failings, and thus scientific fallibility is entwined with human frailty. Some degree of limited scientific replicability is intrinsic to the subject matter itself… At other times, researchers may make unintentional mistakes in interpreting their data or may unknowingly use contaminated samples… However, there are far more egregious errors made by scientists that have a major impact on how science is conducted. There are cases of outright fraud… [but] Such overt fraud tends to be unusual… However, what occurs far more frequently than gross fraud is the gentle fudging of scientific data, consciously or subconsciously, so that desired scientific results are obtained. Statistical outliers are excluded, especially if excluding them helps direct the data in the desired direction. Like most humans, scientists also have biases and would like to interpret their data in a manner that fits with their existing concepts and ideas.
Bravo. This is a major theme of Genesis and Genes, and it is crucial in becoming an informed consumer of science. In this short essay, Rehman obviously cannot describe all the influences that have an impact on scientific research. One of Rehamn’s more important omissions is that there is an enormous amount of conditioning which influences scientists – like everyone else – long before they step into the laboratory. Take evolution. If you grew up in the West any time in the last fifty years, you will have encountered innumerable instances in which the claims of evolutionary biology would have been seared into your consciousness, from David Attenborough documentaries to museum dioramas to advertising campaigns named The evolution of the office to countless articles in New Scientist. Scientists do not enter their research careers with a tabula rasa. As professor John Polkinghorne puts it,
Scientists do not look at the world with a blank gaze; they view it from a chosen perspective and bring principles of interpretation and prior expectations… to bear upon what they observe. Scientists wear (theoretical) “spectacles behind the eyes”.
Human fallibility not only affects how scientists interpret and present their data, but can also have a far-reaching impact on which scientific projects receive research funding or the publication of scientific results. When manuscripts are submitted to scientific journals or when grant proposal are submitted to funding agencies, they usually undergo a review by a panel of scientists who work in the same field and can ultimately decide whether or not a paper should be published or a grant funded. One would hope that these decisions are primarily based on the scientific merit of the manuscripts or the grant proposals, but anyone who has been involved in these forms of peer review knows that, unfortunately, personal connections or personal grudges can often be decisive factors.
Correct. If you happen to be conducting climate research that produces unpopular results, for example, you can be almost sure that your findings will not be published in the most prestigious journals. If you happen to suspect that the brilliant mathematician Irving Segal was right, and that the linear relationship that Edwin Hubble saw between the redshift and apparent brightness of galaxies is perhaps illusory, you are almost certain to receive very little telescope time. Exploring the natural world to your heart’s content, following your curiosity wherever it leads you – that picture of how science was done was fairly accurate up to about the middle of the 19th century. Affluent gentleman scientists could indulge their curiosity about how nature operates. These days, the confines within which research is done will be dictated, to a significant extent, by whatever is considered acceptable by the majority of the community.
The science mystique will eventually topple, and that will be a liberating moment for science. It will usher in an age in which scientists and the public alike will be informed consumers of science, able to accurately assess various findings of scientists and assign to them appropriate levels of credibility.
The post Dr. John Ioannidis and the Reality of Research:
The post Dr. Ben Goldacre and the Reproducibility of Research:
Retrieved 17th May 2013.
 Discovery Institute Press, 2012.
 Dr. Rehman cites this paper at this point:
Retrieved 19th May 2013.
 Dr. Rehman cites the following article:
Retrieved 19th May 2013.