Report of Nature’s peer review trial

Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment.

by Philip Campbell et al.

On 1 June this year, Nature launched a trial of open peer review. The intention was to explore the interest of researchers in a particular model of open peer review, whether as authors or as reviewers. It was also intended to provide Nature’s editors and publishers with a test of the practicalities of a potential extension to the traditional procedures of peer review.

Several times during the exercise, researchers and journalists asked us whether the trial reflected a sense of dissatisfaction or concern about our long-standing procedure. On the contrary, we believe that this process works as well as any system of peer review can. Furthermore, in our occasional surveys of authors we receive strong signals of satisfaction: in the most recent survey, 74% agreed with the statement that their paper had been improved by the process, 20% felt neutral, while 6% disagreed.

Nevertheless, peer review is never perfect and we need to keep it subjected to scrutiny as community expectations and new opportunities evolve. In particular, we felt that it was time to explore a more participative approach.

Continue reading

Ethics: Trust and reputation on the web

William Arms

Online publications have several ways to give themselves a good name.

Trust and reputation are fundamental to scholarly publishing. The web provides tantalizing new ways to publish, but can these win the trust that is crucial to scientific acceptance? Peer review is the traditional way of building trust, but it is slow and expensive; some topics are difficult to review and reviewers miss mistakes. What alternatives do authors have?

Continue reading

Perspective: Peer review of interdisciplinary scientific papers

Christopher Lee

Boundary-crossing research meets border patrol

Both universities and funding agencies have proclaimed the need for more ‘interdisciplinary’ research and more ‘interdisciplinary’ teams, yet publication of such work raises a unique set of challenges for peer review for which many traditional single-discipline journals are not fully prepared. Many of the most exciting research fields today are themselves hybrids of multiple disciplines, yet the peer review system gives rise to problems in assessing interdisciplinary research. The system can be fixed, given the tremendous opportunities today for new approaches

Continue reading

Perspective: The case for group review

Debomoy Lahiri

Peer review would be improved by discussion across the lab

The current individual-based review system has its limitations. Modern research is both multidisciplinary and technical, and it is often difficult to find reviewers that have significant expertise across subfields together with technical know-how in a specific discipline. They are also given only a limited time to complete their review. So reviewers, even those within the same field of research, often differ in their evaluation of a paper. As an alternative, I suggest we use a system which I call peer group review.

Continue reading

Quality and value: Models of quality control for scientific research

Tom Jefferson

A look at alternatives to the peer review system suggests it might be the best bet.

Peer review is used as a quality-control mechanism for biomedical literature (ref. 1). Journals with a formal peer-review system are generally higher profile than those without, and peer review is often a pre-condition for journal indexing in biomedical databases such as PubMed. Variants of the current system or complete alternatives have scarcely been explored, as yet. Two years ago, I and my colleagues carried out an exploratory review of complete and partial alternatives to the current editorial peer-review system, both in and outside biomedical sciences. Here I update and summarise our findings.

Continue reading

Quality and value: The true purpose of peer review?

Charles G. Jennings

What you can’t measure, you can’t manage: the need for quantitative indicators in peer review

Given its importance in steering the global research enterprise, peer review seems under-studied. There is a growing literature on the subject, some of which is highlighted at the quadrennial Peer Review Congress, but for the most part we are still only seeing snapshots. A more systematic approach is needed if we are to understand peer review as it is currently practiced, or to evaluate the pros and cons of any alternative approaches.

Continue reading

Systems: Online frontiers of the peer-reviewed literature

Theodora Bloom

The Internet is allowing much more interactive science publishing

Online tools can be used to improve the accuracy, transparency and usefulness of the scientific literature by moving away from the traditional emphasis on closed peer review. Given the capability for post-publication amendment of articles, the scientific articles themselves and the peer-review process will soon be profoundly different from today’s standard.

Continue reading

Perspective: ‘I don’t know what to believe’

Tracey Brown

Understanding peer review is key to developing informed opinions about scientific research.

The general public are presented with ‘scientific findings’ from a wide range of sources, some more credible than others. Educators complain that pupils and students use web research with little regard for the status of what they find. Medical helplines are inundated with calls about risks and cures following media stories. And, much to the frustration of scientists, unwarranted scares, pseudoscience and health fads abound. How can judgements be made and useful questions asked?

Continue reading

Technical solutions: Evolving peer review for the internet

Richard Akerman

Peer review needs to adapt to the pace and volume of information published online

How does the role of peer review evolve when the body of scholarly knowledge expands from slowly circulating, static documents to the universe of rushing, dynamic interactions made possible by the Internet? Although traditional forms of scholarly communication are still used, the sheer volume and pace of information enabled by the Internet and publishing tools such as weblogs (blogs) demands novel solutions.

Continue reading

Ethics: Detecting misconduct

Dale Benos

Does a digital workflow make it easier to detect ethical breeches in peer review?

The Internet has changed everything. You can be sitting at your desk in Birmingham, Alabama, while having a conversation in real time with a colleague in Birmingham, United Kingdom, exchanging not only words and ideas, but also photographs, data sets and manuscripts. The Internet has also changed the way science is done, particularly when it comes to publication. Manuscripts are now submitted, reviewed and authors notified electronically. But although the efficiency and speed of the peer-review process has increased, a set of attendant issues has arisen.

Continue reading