Nature peer-review editorial popular with readers

The most downloaded PDFs from the Nature‘s website for December 2006 show the Editorial “Peer Review and Fraud” at number 6, even though the article was published in the last issue of the year — traditionally disadvantageous for a high rank in the monthly download statistics. The Editorial was accompanied by a report on Nature’s Peer Review trial, which is also posted on Peer-to-Peer.

Other popular PDF downloads for December included a Correspondence contribution from Prof U. Kutschera about creationism and evolution; the news story “In search of lost time”; and, and number 1, the Letter “Decoding the ancient Greek astronomical calculator known as the Antikythera Mechanism.” The full list is available at Nature’s “top ten” page.

“Content Matters” on Nature’s peer review trial

Barry Graubart writes about Nature‘s peer-review trial at his Content Matters blog. Mr Graubart refers to the Wall St Journal’s erroneous comment that Nature has “cancelled” the trial. As explained in the Nature report , the trial was originally intended to be a three-month experiment starting in June. In the event, we extended it by a month, closing it to new submissions in October. Since then, the remaining manuscripts in the trial have completed the peer-review process and we have been analysing the results (which necessarily meant waiting until the final manuscripts had received referees’ reports and could be removed from the trial).

Mr Graubart writes: “While it’s disappointing that this experiment did not succeed, the fact that they tried this open peer review process is a testament to the team at Nature. I have previously posted about Nature‘s position as an innovator among content providers. While this experiment may not have produced the results they’d hoped for, I have no doubt they will continue to push the envelope in testing new technologies.”

We’re grateful for those kind words, and are sure that Mr Graubart’s final suggestion is correct.

Richard Charkin on Nature’s peer-review trial

Richard Charkin, Chief Executive of Macmillan, owner of Nature Publishing Group, has posted an interesting entry on his blog about Nature’s peer review trial. The post contains a link to a radio interview with Philip Campbell, Editor in Chief of Nature, and features an article on the trial in the Wall St Journal.

Mr Charkin writes: “It’s great that a subject so apparently arcane as scientific peer review should be considered important enough to warrant two slots on the most important radio programme in the UK and a feature in the world’s leading financial newspaper. What is not so great is that the discussions manage to confuse open reviewing with free access, comment with criticism, freedom of information with free information, an excellent system which catches nearly all attempted scientific fraud with a flawed system which allows fraud to happen, the desire to speak confidentially and openly as opposed to the apparently open but necessarily guarded alternative. In other words and as usual, a tricky and important debate has been reduced to a few soundbites of little value and significant distortion.”

Report of Nature’s peer review trial

Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment.

by Philip Campbell et al.

On 1 June this year, Nature launched a trial of open peer review. The intention was to explore the interest of researchers in a particular model of open peer review, whether as authors or as reviewers. It was also intended to provide Nature’s editors and publishers with a test of the practicalities of a potential extension to the traditional procedures of peer review.

Several times during the exercise, researchers and journalists asked us whether the trial reflected a sense of dissatisfaction or concern about our long-standing procedure. On the contrary, we believe that this process works as well as any system of peer review can. Furthermore, in our occasional surveys of authors we receive strong signals of satisfaction: in the most recent survey, 74% agreed with the statement that their paper had been improved by the process, 20% felt neutral, while 6% disagreed.

Nevertheless, peer review is never perfect and we need to keep it subjected to scrutiny as community expectations and new opportunities evolve. In particular, we felt that it was time to explore a more participative approach.

Continue reading