Peer-to-peer

William Arms

Online publications have several ways to give themselves a good name.

Trust and reputation are fundamental to scholarly publishing. The web provides tantalizing new ways to publish, but can these win the trust that is crucial to scientific acceptance? Peer review is the traditional way of building trust, but it is slow and expensive; some topics are difficult to review and reviewers miss mistakes. What alternatives do authors have?

Continue reading "Ethics: Trust and reputation on the web" »

Posted by Maxine Clarke on June 29, 2006
Categories: Ethics, Peer review debate | Permalink | Comments (0)

Christopher Lee

Boundary-crossing research meets border patrol

Both universities and funding agencies have proclaimed the need for more ‘interdisciplinary’ research and more ‘interdisciplinary’ teams, yet publication of such work raises a unique set of challenges for peer review for which many traditional single-discipline journals are not fully prepared. Many of the most exciting research fields today are themselves hybrids of multiple disciplines, yet the peer review system gives rise to problems in assessing interdisciplinary research. The system can be fixed, given the tremendous opportunities today for new approaches

Continue reading "Perspective: Peer review of interdisciplinary scientific papers" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (2)

Debomoy Lahiri

Peer review would be improved by discussion across the lab

The current individual-based review system has its limitations. Modern research is both multidisciplinary and technical, and it is often difficult to find reviewers that have significant expertise across subfields together with technical know-how in a specific discipline. They are also given only a limited time to complete their review. So reviewers, even those within the same field of research, often differ in their evaluation of a paper. As an alternative, I suggest we use a system which I call peer group review.

Continue reading "Perspective: The case for group review" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (0)

Tom Jefferson

A look at alternatives to the peer review system suggests it might be the best bet.

Peer review is used as a quality-control mechanism for biomedical literature (ref. 1). Journals with a formal peer-review system are generally higher profile than those without, and peer review is often a pre-condition for journal indexing in biomedical databases such as PubMed. Variants of the current system or complete alternatives have scarcely been explored, as yet. Two years ago, I and my colleagues carried out an exploratory review of complete and partial alternatives to the current editorial peer-review system, both in and outside biomedical sciences. Here I update and summarise our findings.

Continue reading "Quality and value: Models of quality control for scientific research" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Quality and value | Permalink | Comments (1)

Charles G. Jennings

What you can’t measure, you can’t manage: the need for quantitative indicators in peer review

Given its importance in steering the global research enterprise, peer review seems under-studied. There is a growing literature on the subject, some of which is highlighted at the quadrennial Peer Review Congress, but for the most part we are still only seeing snapshots. A more systematic approach is needed if we are to understand peer review as it is currently practiced, or to evaluate the pros and cons of any alternative approaches.

Continue reading "Quality and value: The true purpose of peer review?" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Quality and value | Permalink | Comments (0)

Theodora Bloom

The Internet is allowing much more interactive science publishing

Online tools can be used to improve the accuracy, transparency and usefulness of the scientific literature by moving away from the traditional emphasis on closed peer review. Given the capability for post-publication amendment of articles, the scientific articles themselves and the peer-review process will soon be profoundly different from today’s standard.

Continue reading "Systems: Online frontiers of the peer-reviewed literature" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Systems | Permalink | Comments (2)

Tracey Brown

Understanding peer review is key to developing informed opinions about scientific research.

The general public are presented with 'scientific findings' from a wide range of sources, some more credible than others. Educators complain that pupils and students use web research with little regard for the status of what they find. Medical helplines are inundated with calls about risks and cures following media stories. And, much to the frustration of scientists, unwarranted scares, pseudoscience and health fads abound. How can judgements be made and useful questions asked?

Continue reading "Perspective: 'I don't know what to believe'" »

Posted by Maxine Clarke on June 20, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (3)

Richard Akerman

Peer review needs to adapt to the pace and volume of information published online

How does the role of peer review evolve when the body of scholarly knowledge expands from slowly circulating, static documents to the universe of rushing, dynamic interactions made possible by the Internet? Although traditional forms of scholarly communication are still used, the sheer volume and pace of information enabled by the Internet and publishing tools such as weblogs (blogs) demands novel solutions.

Continue reading "Technical solutions: Evolving peer review for the internet" »

Posted by Maxine Clarke on June 20, 2006
Categories: Peer review debate, Technical solutions | Permalink | Comments (0)

Dale Benos

Does a digital workflow make it easier to detect ethical breeches in peer review?

The Internet has changed everything. You can be sitting at your desk in Birmingham, Alabama, while having a conversation in real time with a colleague in Birmingham, United Kingdom, exchanging not only words and ideas, but also photographs, data sets and manuscripts. The Internet has also changed the way science is done, particularly when it comes to publication. Manuscripts are now submitted, reviewed and authors notified electronically. But although the efficiency and speed of the peer-review process has increased, a set of attendant issues has arisen.

Continue reading "Ethics: Detecting misconduct" »

Posted by Maxine Clarke on June 20, 2006
Categories: Ethics, Peer review debate | Permalink | Comments (3)

Trish Groves

A recipe for good peer review

Improving peer review depends on making its human aspects more humane. Journals need to ask the right reviewers to review the right articles, help them to do it quickly and thoroughly, make them feel happy to sign their reports, thank them, tell them how they did, and encourage wide recognition of what’s too often a thankless task.

Continue reading "Quality and value: How can we get the best out of peer review?" »

Posted by Maxine Clarke on June 20, 2006
Categories: Peer review debate, Quality and value | Permalink | Comments (0)

Erik Sandewall

A hybrid system of peer review

Traditional peer review serves two purposes: to give feedback to the authors, helping them to improve their manuscript; and to control the quality of published articles. I believe that more value can be obtained by incorporating an open element in the peer-review system.

Continue reading "Systems: Opening up the process" »

Posted by Maxine Clarke on June 20, 2006
Categories: Peer review debate, Systems | Permalink | Comments (0)

Brenda Riley

Database publication presents unique challenges for the peer reviewer

The reader of a scientific paper in a high-quality journal knows that the information has been vetted by a formal process of peer review, moderated by editors. But the face of publishing is changing, and peer review of databases is becoming an increasingly important facet of scientific data curation. The Signaling Gateway's molecule pages represent an important, innovative experiment in applying models of peer review developed in journals to the much newer world of scientific databases.

Continue reading "Systems: Trusting data's quality" »

Posted by Maxine Clarke on June 20, 2006
Categories: Peer review debate, Systems | Permalink | Comments (0)

Thomas DeCoursey

Should authors be told who their reviewers are?

The goal of any change in the peer review system must be to improve the quality of review, where quality is determined by two distinct functions: filtering manuscripts for publication in a given journal; and making constructive suggestions on how the manuscript or study could be improved. Would open review (in which reviewers sign their reviews) accomplish this goal? I have experienced several cases of open review, intentional and unintentional, with mixed results.

Continue reading "Perspective: The pros and cons of open peer review" »

Posted by Maxine Clarke on June 14, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (9) | TrackBacks (1)

Chris Anderson

Scientific publishers should let their online readers become reviewers.

Who are the peers in peer review? In journals such as Nature, they usually have a PhD and work in a field relevant to the paper under consideration. If they are academics, they may be tenured professors, usually people on a relatively short list of experts who have agreed to review papers. This is a little élitist, but credentials such as PhDs and tenure are given in part to reward those things – experience, insight, brains and the respect of other researchers – that also make for wise advice. The process is not perfect, for reasons ranging from cronyism to capriciousness, yet long experience has shown it to be better than the alternatives.

Continue reading "Technical solutions: Wisdom of the crowds" »

Posted by Maxine Clarke on June 14, 2006
Categories: Peer review debate, Technical solutions | Permalink | Comments (9)

Elizabeth Wager

Analysing the purpose of peer review.

Most people accept that peer review is enormously valuable and should be maintained and protected, but few agree on what purpose it serves. Science publishing, even at its simplest, involves complex interactions between researchers (authors), journal editors, reviewers and readers. It is a subtle form of human behaviour that could furnish the raw material for dozens of sociology theses. And when academic endeavour gets mixed up with commercial interests, things get even more complicated.

Continue reading "Ethics: What is it for?" »

Posted by Maxine Clarke on June 14, 2006
Categories: Ethics, Peer review debate | Permalink | Comments (0)

David Ozonoff

Researchers need reviewers to check their stats.

Statistical methods are widely used in many areas of natural science, especially in my field of research, epidemiology. Although statistical procedures are often viewed as a black art, or as a black box, they are not limited to specialists. With today’s computing power and software, researchers can and do use computationally intensive methods of great complexity, often leading to the use of techniques that are more sophisticated and powerful than necessary. Many researchers have trouble interpreting the results, or interpret them incorrectly. Clearly, this is a matter for peer review.

Continue reading "Quality and value: Statistics in peer review" »

Posted by Maxine Clarke on June 14, 2006
Categories: Peer review debate, Quality and value | Permalink | Comments (4)

Thomas Koop and Ulrich Pöschl

The editors of Atmospheric Chemistry and Physics explain their journal’s approach.

Recent high-profile cases of scientific fraud have fuelled the discussion of scientific quality control. A problem of similar, if not greater, importance is the large proportion of carelessly prepared scientific papers that dilute rather than enhance scientific knowledge. Both problems indicate shortcomings in the traditional peer-review system. Many scientists and publishers believe that peer review remains the best available approach for quality assurance, but requests for improvements are commonplace.

Continue reading "Systems: An open, two-stage peer-review journal" »

Posted by Maxine Clarke on June 14, 2006
Categories: Peer review debate, Systems | Permalink | Comments (2)

John Moore

Even reviewed literature can be cherry-picked to support any argument.

The research community understands that scientific information that has not been peer reviewed should not be taken seriously. As scientists, we discriminate between what is put out on blogs or in press releases and what is published in the formal scientific literature. We also know the difference between a peer-reviewed primary paper or review, and an unreviewed letter to the editor or opinion piece. In other words, we understand the peer-review system, and use it as a filter to sort the wheat from the chaff.

Continue reading "Perspective: Does peer review mean the same to the public as it does to scientists?" »

Posted by Maxine Clarke on June 05, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (3) | TrackBacks (1)

Herbert Van de Sompel

What functions do we take for granted in print?

The Digital Library Research and Prototyping Team at the research library of the Los Alamos National Laboratory conducts research on various aspects of scholarly communication in the digital age, including peer review. Our research attempts simultaneously to analyse properties of the existing review system, and to formulate feasible alternatives.

Continue reading "Technical solutions: Certification in a digital era" »

Posted by Maxine Clarke on June 05, 2006
Categories: Peer review debate, Technical solutions | Permalink | Comments (1)

Kirby Lee and Lisa Bero

What authors, editors and reviewers should do to improve peer review.

Peer review is not currently designed to detect deception, nor does it guarantee the validity of research findings. It should, however, identify flaws in the design, presentation, analysis and interpretation of science and provide prompt, detailed, constructive criticism to improve research.

Continue reading "Ethics: Increasing accountability" »

Posted by Maxine Clarke on June 05, 2006
Categories: Ethics, Peer review debate | Permalink | Comments (7)

Joan E. Sieber

Improving the peer-review process relies on understanding its context and culture.

Peer review gets a bad rap these days, and there is much talk that something should be done about it. Scientific societies need to know more about the alleged problems before they try to fix the peer-review process. These problems are often cast as ethical ones: unintended bad consequences of well-meaning editorial processes.

Continue reading "Quality and value: How can we research peer review?" »

Posted by Maxine Clarke on June 05, 2006
Categories: Peer review debate, Quality and value | Permalink | Comments (3) | TrackBacks (2)

Eugene Koonin, Laura Landweber, David Lipman and Ros Dignon

Can 'open peer review' work for biologists? Biology Direct is hopeful.

The advent of immensely powerful means of communication in our information age offers unprecedented opportunities for experimentation with new approaches to scientific publishing. In an attempt to offer the scientific community an alternative to the current peer-review system, we recently launched a new journal, Biology Direct.

Continue reading "Systems: Reviving a culture of scientific debate" »

Posted by Maxine Clarke on June 05, 2006
Categories: Peer review debate, Systems | Permalink | Comments (1) | TrackBacks (1)