Peer-to-Peer

Systems: An open, two-stage peer-review journal

Thomas Koop and Ulrich Pöschl

The editors of Atmospheric Chemistry and Physics explain their journal’s approach.

Recent high-profile cases of scientific fraud have fuelled the discussion of scientific quality control. A problem of similar, if not greater, importance is the large proportion of carelessly prepared scientific papers that dilute rather than enhance scientific knowledge. Both problems indicate shortcomings in the traditional peer-review system. Many scientists and publishers believe that peer review remains the best available approach for quality assurance, but requests for improvements are commonplace.


Some suggest that reviewers’ comments and authors’ replies should be publicly exchanged1,2, and that manuscripts should be openly discussed before formal publication3,4. We believe that these ideas and their advantages can be efficiently combined with the strengths of traditional peer review.

Quality assurance of scientific publications usually proceeds through two pathways: a pre-publication short-term assessment by designated reviewers during the peer-review process, and a post-publication long-term assessment by the scientific community through comments, citations, review articles and monographs. Both can be combined in a collaborative peer-review process where members of the scientific community participate in the assessment of scientific manuscripts through interactive comments, in addition to designated reviewers’ reports.

The journal

This approach is pursued by the open-access journal Atmospheric Chemistry and Physics and a growing number of sister journals published by the European Geosciences Union. These journals have a two-stage publication process with public peer review and interactive discussion5.

In the first stage, manuscripts that pass a rapid pre-screening (access review) are immediately published as ‘discussion papers’ on the journal’s website. They are then subject to interactive public discussion for a period of 8 weeks, during which the comments of designated reviewers, additional comments by other interested members of the scientific community, and the authors’ replies are published alongside the discussion paper. Reviewers can choose to sign their comments or remain anonymous, but comments by other scientists must be signed.

In the second stage, manuscript revision and peer review are completed in the same way as in traditional journals (with further rounds of non-public revisions and review where required). If accepted, final papers are published in the main journal. To provide a lasting record of review and to secure the authors’ publication precedence, every discussion paper and interactive comment remains permanently archived and individually citable.

The results

Atmospheric Chemistry and Physics was established in 2001 and last year published about 240 final papers. On average, one in four papers receives a comment from the scientific community in addition to the comments from designated reviewers (in traditional journals this happens for about one in 100 papers). And there are typically 0.45 pages of comments and replies per page of original discussion paper. These cover the full spectrum of opinions, from harsh criticism to open applause, and provide a wealth of additional information and evaluation that is available to everyone.

Our statistics confirm that collaborative peer review facilitates and enhances quality assurance. The journal has a relatively low overall rejection rate of less than 20%, but only three years after its launch the ISI journal impact factor ranked Atmospheric Chemistry and Physics twelfth out of 169 journals in ‘Meteorology and Atmospheric Sciences’ and ‘Environmental Sciences’.

These numbers support the idea that public peer review and interactive discussion deter authors from submitting low-quality manuscripts, and thus relieve editors and reviewers from spending too much time on deficient submissions.

The advantages

The deterrent is particularly important, because reviewing capacities are the most limited resource in the publication process. Although peer review depends crucially on the availability and performance of reviewers, it has traditionally offered little reward for those providing careful and constructive reviews. In public review, however, reviewers’ arguments are publicly heard and, if comments are openly signed, reviewers can also claim authorship for their contribution.

Over five years at Atmospheric Chemistry and Physics, we have found that about two-thirds of our reviewers prefer to remain anonymous. There are, however, interesting differences between subdisciplines: around 50% of modellers sign their reviewer comments, whereas only 30% of the experimentalists do so. It seems that modellers more often provide suggestions and ideas for which they like to claim authorship as a reward.

We believe that collaborative peer review with a two-stage publication process and interactive public discussion effectively resolves the dilemma between rapid scientific exchange and thorough quality assurance. It fosters scientific discussion, deters submission of substandard manuscripts, conserves reviewing capacities, and enhances the density of information in final papers. Moreover, it can be flexibly integrated into existing journals as well as large-scale publishing systems and repositories (such as ArXiv), simply by adding an interactive discussion forum.

REFERENCES

1. Lahiri, D. K. Nature 439, 784 (2006).

2. Kennedy, D. Science 312, 159 (2006).

3. Van Meir, E. G. Nature 429, 803 (2004).

4. de Carvalho, L. B. Nature 439, 784 (2006).

5. Pöschl, U. Learned Publ. 17, 105 (2004).

Thomas Koop is professor of physical chemistry in the department of chemistry at Bielefeld University, Germany, and an executive editor of Atmospheric Chemistry and Physics.

Ulrich Pöschl is head of a research group at the Max Planck Institute for Chemistry in Mainz, Germany. He is also initiator and executive editor of Atmospheric Chemistry and Physics, and president of the Atmospheric Sciences Division of the European Geosciences Union.

Read more See this article in Nature’s web focus here

Comments

  1. Dinesh Kumar Singh said:

    This is an excellent suggestion and a wise way of reviewing the papers.The only addition it needs is to keep the name of authors,insitute from which it is sent for review anonymous during the review process.Also, the name of the people posting comments should be kept anonymous during this period.I think it will greatly increase the quality of scientific papers being published.

  2. Lenin Domínguez said:

    I like the idea of the two-stage peer review. I was worried about the use and abuse of the information (that is, the article itself) to third-parties (ie, the public reviewers). The description of the process at the “Atmospheric Chemistry and Physics” is very well executed.

    I am glad there are journals making an effort to change and improve the way scientific papers are reviewed.

    Report this comment Cancel report
    Your details

    Please confirm the words below

    In order to reduce spamming, this process ensures you are a real person and not an automated program.

Comments are closed.