Science Philosophy and Practice: Science Communications and Peer Review

Introduction

Peer review is the process by which scholarly articles and the research they contain are evaluated for accuracy and relevance by qualified experts. The scientific community employs peer review as a way to evaluate whether articles should be published in journals. A similar system is used to determine whether applications for research grants should be approved.

The strength of the peer review process lies in the experts, sometimes employed as journal editors, who evaluate the submitters' work. Expert evaluation encourages authors to meet commonly accepted standards of quality in their research and the conclusions they draw from it. The most effective evaluators are often those who are actively conducting research in the same field. However, to ensure neutrality, the identity of an article's author is often concealed during the peer review process. Some journals, such as the prestigious Nature, view peer review as a technique for improving articles and the journal as a whole.

Historical Background and Scientific Foundations

Today, peer review is considered essential to the credibility of an article. Just as universities and schools must be accredited to issue degrees that are respected, scientific journals must conduct peer review to ensure the merit of their articles. Though it is a requirement today, peer review was not formalized or used consistently until the middle of the twentieth century.

Medicine was the first field to employ systematic peer review and declare it to be an essential method of quality control. In the mid-ninth century, the Syrian physician Ishap bin Ali al-Rahwi (854–931) described a peer review system in which physicians were required to keep notes on their patients and treatments. In the case of a death, a panel of other physicians would evaluate whether the treatment was correct and whether the treating physician could be sued.

As the sciences developed, peer review was present in an informal way through the relationships among individual scientists. Colleagues and friends evaluated each others' work as part of the collaborative process, sometimes in association with scientific academies. Scientific controversies were often vigorously debated among the members of academies, improving the state of science through the application of the scientific method. This system of peer review was not a formal evaluation of scientific papers, but an open dialogue between qualified experts.

In the early twentieth century, formal scientific journals were well established. Often under the authority of a single expert editor, articles would be accepted or rejected based on the editor's sole evaluation. The Nobel Prize—winning physicist Max Planck (1858–1947),

editor of the journal Annalen der Physik, was a good example of this model. As one of the leading researchers of his day, Planck was active in the field of physics, constantly reading research papers, and acquainted with the different kinds of research going on at the time. However, the publishing industry could not always secure editors of such high quality. This led to the development of the more collaborative modern system of peer review.

Modern Cultural Connections

Today peer review is the standard by which scientific research is judged for quality. Research, articles, or claims that do not withstand the scrutiny of peer review are generally considered flawed or incorrect. Peer review varies among the many different scientific journals, but generally editors seek the advice of expert reviewers to better determine the quality of articles they consider for publication. In the classic blind review method, the reviewers' and authors' identities are concealed from one another to prevent conflicts of interest or other unethical behavior. The reviewers must read and provide comments to the editor in a timely manner to avoid unnecessary delay. This system is praised for excluding poor-quality work and for helping avoid duplicate publications of similar work.

Despite the system's broad acceptance, criticisms are numerous. Though peer review is widely believed to improve the quality of published papers, there is no measurable evidence that yet confirms this claim. Also, the personalities and egos of the reviewers and authors can interfere with the neutrality of the process, particularly when they are rivals in the same field. Other critics allege that many reviewers have an unconscious bias toward orthodox or traditional thinking and wrongly reject unconventional approaches in research. Recently, major cases of fraud have called the effectiveness of peer review into question. One example is that of the South Korean stem cell researcher Hwang Woo-Suk (1953–), whose articles, published in the journal Science, contained large amounts of fabricated data. The inability to detect even large-scale fraud is a much-discussed weakness of modern peer review.

Editors try to minimize these weaknesses by introducing elements of openness to the review and selection of articles. Several well-known scientific journals, including Nature and the Australian Medical Journal, have experimented with the use of an online forum as part of their peer review processes. The system at Nature subjected approved articles to both traditional peer review and the new online approach simultaneously. Subscribers registered at the journal's Web site were invited to participate, limiting input from inexpert reviewers. The comments were moderated by the journal and reviewers were required to identify themselves. Though many people read the articles online, few actually commented

on them, and even fewer of these comments were judged to be helpful to the editors or of similar quality to the traditional blind reviewers' comments. Because of these problems, editors at Nature abandoned this system.

Other journals have had more success and continue to use online, open peer review to improve their articles.

This initial failure does not mean that openness in peer review or the use of the Internet to include more reviewers will ultimately be rejected. Serious problems have arisen in peer-reviewed journal articles despite the proper application of the current system. Members of the scientific community are trying to determine the best methods for improving peer review. Success, whether it is measured in increased openness, improved analysis, or other techniques, should improve science overall and help direct researchers to the best research methods.

Primary Source Connection

The Internet now serves as an efficient mechanism for the scientific community to share vital information.

Several major universities are pondering and experimenting with the merits of open publication of scientific papers on the Web as an alternative to requiring that faculty strictly publish in often expensive traditional journals.

The Internet has also become an essential tool for scientists to communicate and share information when time is critical, especially during outbreaks of disease. In the following article, virologist Jack Woodall recounts the founding of ProMed mail, a Web-based forum where clinicians or researchers in the field can report data and observations of disease outbreaks from any spot on the globe to a centralized point where fellow experts can instantaneously review and respond. In 2003, the outbreak eventually identified as SARS was first reported to ProMED by a physician who learned from a colleague via e-mail that an unusual pneumonia was killing people and taxing the resources of hospitals in the Guangzhou region of China.

Woodall, who was an original founder of ProMed mail, continues to serve as its associate editor. Woodall is a graduate of Cambridge University in the United Kingdom, and he received his Ph.D. from London University. During a long and distinguished career, Woodall has served and worked at the East African Virus Research Institute in Uganda, the Belem Virus Laboratory in Brazil, and the Yale Arbovirus Research Unit in New Haven, Connecticut. He has also served as the head of the Arbovirus Laboratory for the New York State Health Department and as director of the Centers for Disease Control and Prevention's San Juan Laboratories in Puerto Rico. In 1981, he began work for the World Health Organization (WHO) and was a member of the WHO Gulf Emergency Task Force in support of the U.N. Special Commission (UNSCOM) in Iraq, and leader of the WHO delegation to the Third Review Conference on the Biological Weapons Convention. Until 2007, he served as visiting professor at the Institute of Medical Biochemistry and as director of the Nucleus for the Investigation of Emerging Infectious Diseases at the Federal University of Rio de Janeiro, Brazil.

In addition to his continued work with ProMED mail, Woodall is a member of the International Society for Infectious Diseases (ISID), and Web site editor and council member (ex officio) of the American Society of Tropical Medicine and Hygiene. He is a member of the Biological Weapons Working Group of the Center for Arms Control and Non-Proliferation and a board member for Sabin Vaccine Institute in Washington D.C.

PROMED MAIL

ProMED mail, to give it its baptismal name, or ProMED, as everyone now calls it, was a happy accident. Barbara Hatch Rosenberg of the Federation of American Scientists organized a meeting in 1993 in Geneva, Switzerland, co-sponsored by the World Health Organization (WHO), to float the idea of a world-girdling chain of institutes capable of sending out teams to the site of any unusual disease outbreak in their neighborhood. The objective would be to determine whether it was of natural or unnatural origin. The conference itself was unusual in bringing together experts on not only human, but also animal and plant diseases, and on bioterrorism, which at the time was not high on anyone's priority list.

There were some 60 participants from 15 countries. The conclusion was that such a chain was highly desirable from the point of view of human health and food security. At a follow-up conference in the United States in 1994, further steps were outlined, and the late Dr. Robert Shope suggested the name ProMED, for Program for Monitoring Emerging Diseases. It was decided that an e-mail list be set up to enable discussion among the participating institutions. Charles Clemens of Satel-Life offered to host the e-mail list, and I offered to run it, with the assistance of Stephen Morse, then of Rockefeller University. I was working for the New York State Health Department in Albany, New York, at the time, and was one of only a few of the conference participants who had access to e-mail then. Thus, ProMED-mail, so called to distinguish it from its parent program, was launched in August 1994.

It turned out that absolutely no one in the program had anything to say to each other, so as we were supposed to be monitoring outbreaks, Steve and I started posting outbreak reports from the media. Then, in May 1995, the Ebola epidemic in Kikwit, Zaire, hit the media, and people surfing the Web discovered that ProMED was posting information about it. I well remember the thrill when our mailing list, which had begun with 40 members in seven countries, hit 250. Later we were written up in the Wall Street Journal and our numbers went overnight to 500. Today, in mid-2007, we stand at over 40,000 in at least 180 countries, with many more accessing our Web site at http://www.promedmail.org. And, thanks to foundations and donations, we still provide worldwide coverage, 7/365, without fee.

The uniqueness of ProMED is its stable of experts in the fields of clinical and veterinary medicine, microbiology, and plant pathology, all of whom serve on a part-time basis. It is the only free disease reporting system to cover human, livestock, wildlife, and food and feed crop diseases in one place, the latter because of the potential impact of animal and vegetable diseases on nutrition and therefore on human health. Since 2000, ProMED has been a program of the International Society of Infectious Diseases, which guarantees its freedom from political constraints that often cause delays in outbreak reporting. In fact, WHO [World Health Organization] has said that it uses ProMED reports to convince recalcitrant countries to report outbreaks officially, in view of the fact that a report has already appeared on ProMED.

During the anthrax-by-mail episode in the United States in 2001, the Science Advisor to the President told us that the White House's main sources of updates on the situation were CNN and ProMED. The Chief Veterinary Officers of Australia and New Zealand routinely copy livestock outbreak reports to ProMED at the same time as they send them to the World Animal Health Organization, and we get reports directly from hospitals and research institutes involved in outbreaks. We emphasize reports on outbreaks caused by select agents from the bioterrorism A list, such as anthrax and botulinum toxin, so that our readership understands that such natural outbreaks are not uncommon in some countries. We cover outbreaks due to biological toxins. Otherwise, we report on emerging diseases such as bird flu, using a rather broad definition of emerging that includes dengue but excludes most tuberculosis and HIV/AIDS reports.

ProMED has parallel lists in Spanish, Portuguese, and Russian, with a French version scheduled to launch shortly. These are not straight translations of the English reports, but are mainly reports of regional interest. Chinese and Japanese translations of many ProMED reports are found on the travel health Web sites of Hong Kong and Tokyo International Airport.

Jack Woodall

woodall, jack. “promed mail.” in infectious diseases: in context. edited by br enda w ilmoth lerner and k. lee lerner. detroit: gale, 2007.

See Also Science Philosophy and Practice: Professionalization; Science Philosophy and Practice: Research Funding and the Grant System; Science Philosophy and Practice: Scientific Academies, Institutes, Museums, and Societies.

bibliography

Books

Woodall, Jack. “ProMed Mail.” In Infectious Diseases: In Context. Edited by Brenda Wilmoth Lerner and K. Lee Lerner. Detroit: Gale, 2007.

Periodicals

Gitanjali, B. “Peer Review—Process, Perspectives, and the Path Ahead.” Journal of Postgraduate Medicine 47 (2001): 210–214.

Greaves, Sarah. “Nature 's Trial of Open Peer Review.” Nature (December 2006).

Judson, Horace Freeland. “Structural Transformations of the Sciences and the End of Peer Review.” Journal of the American Medical Association 272 (1994): 92–94.

Spier, Ray. “The History of the Peer-Review Process.” Trends in Biotechnology 20 (August 8, 2002).

Web Sites

International Society of Infectious Diseases. “ProMED.” http://www.promedmail.org (accessed February 5, 2008).

Nature. “Peer Review: Debate.” December 2007. http://www.nature.com/nature/peerreview/debate/index.html (accessed January 2008).

Kenneth T. LaPensee