Peer-to-peer

Barry Graubart writes about Nature's peer-review trial at his Content Matters blog. Mr Graubart refers to the Wall St Journal's erroneous comment that Nature has "cancelled" the trial. As explained in the Nature report , the trial was originally intended to be a three-month experiment starting in June. In the event, we extended it by a month, closing it to new submissions in October. Since then, the remaining manuscripts in the trial have completed the peer-review process and we have been analysing the results (which necessarily meant waiting until the final manuscripts had received referees' reports and could be removed from the trial).

Mr Graubart writes: "While it's disappointing that this experiment did not succeed, the fact that they tried this open peer review process is a testament to the team at Nature. I have previously posted about Nature's position as an innovator among content providers. While this experiment may not have produced the results they'd hoped for, I have no doubt they will continue to push the envelope in testing new technologies."

We're grateful for those kind words, and are sure that Mr Graubart's final suggestion is correct.

Posted by Maxine Clarke on December 21, 2006
Categories: Peer review trial | Permalink | Comments (0)

Richard Charkin, Chief Executive of Macmillan, owner of Nature Publishing Group, has posted an interesting entry on his blog about Nature's peer review trial. The post contains a link to a radio interview with Philip Campbell, Editor in Chief of Nature, and features an article on the trial in the Wall St Journal.

Mr Charkin writes: "It's great that a subject so apparently arcane as scientific peer review should be considered important enough to warrant two slots on the most important radio programme in the UK and a feature in the world's leading financial newspaper. What is not so great is that the discussions manage to confuse open reviewing with free access, comment with criticism, freedom of information with free information, an excellent system which catches nearly all attempted scientific fraud with a flawed system which allows fraud to happen, the desire to speak confidentially and openly as opposed to the apparently open but necessarily guarded alternative. In other words and as usual, a tricky and important debate has been reduced to a few soundbites of little value and significant distortion."

Posted by Maxine Clarke on December 21, 2006
Categories: Peer review trial | Permalink | Comments (0)

Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment.
by Philip Campbell et al.

On 1 June this year, Nature launched a trial of open peer review. The intention was to explore the interest of researchers in a particular model of open peer review, whether as authors or as reviewers. It was also intended to provide Nature's editors and publishers with a test of the practicalities of a potential extension to the traditional procedures of peer review.

Several times during the exercise, researchers and journalists asked us whether the trial reflected a sense of dissatisfaction or concern about our long-standing procedure. On the contrary, we believe that this process works as well as any system of peer review can. Furthermore, in our occasional surveys of authors we receive strong signals of satisfaction: in the most recent survey, 74% agreed with the statement that their paper had been improved by the process, 20% felt neutral, while 6% disagreed.

Nevertheless, peer review is never perfect and we need to keep it subjected to scrutiny as community expectations and new opportunities evolve. In particular, we felt that it was time to explore a more participative approach.

Continue reading "Report of Nature's peer review trial" »

Posted by Maxine Clarke on December 20, 2006
Categories: Peer review debate, Peer review trial | Permalink | Comments (10)

At Nautilus, the blog for authors, I've posted about competing interest statements in scientific papers. I've asked there what scientists think of this practice as authors. But what about when the scientists are peer-reviewers? Does it make a difference to you, when you are reviewing a paper, to know that the author has taken out a patent on the discovery or has shares in a spinoff company? Do peer-reviewers judge the scientific results independently of these declarations, or does it make a difference to the level of scrutiny they apply to the work? The Nature journal editors would like to hear about the reviewers' perspective, via your comments to this post.

Posted by Maxine Clarke on December 19, 2006
Categories: Ethics, Quality and value | Permalink | Comments (0)

Peer-to-Peer and Nautilus have received their first welcome to the scientific blogosphere from outside the Nature Publishing Group. At Blog Around the Clock, Coturnix, er, welcomes us here. Lukman has added a comment: "Thanks for this great info. Your blog is very informative for science development for ordinary people like us."

In an earlier post, Coturnix writes about the upcoming 2007 North Carolina Science Blogging conference, part of the annual meeting of the Society for Integrative and Comparative Biology. From the society's website: "Popular political and social commentary blogs are making the news, but is there more out there than chatty gossip and collections of links? How about some science? Can this trendy technology be useful for scientists? Come to the Media Workshop and find out ....about how blogging works, setting one up, finding things to write about, and using the medium for your classes, for research, or for educating the public."

Correction: The session on science blogging at SCIB is on January 4th in Phoenix, AZ, while the 1.5-day Science Blogging Conference is on January 20th in Chapel Hill, NC. (Thanks to Coturnix for pointing this out.)

Posted by Maxine Clarke on December 13, 2006
Categories: General | Permalink | Comments (4)

Q. I am writing to ask you about the minimal standards required by your journal concerning statistical analyses in submitted manuscripts.

I am a biostatistician. A colleague, a biomedical researcher, insists that there are no sample size requirements as long as the minimum statistical power is achieved. To be more concrete, consider a simple two-group t-test where each group is of size n=3 and the power calculations show that the power=80% (a widely accepted standard). My argument is that statistics on sample size of n=3 per group does not make much sense in terms of reliability, but my colleague insists that because the power is 80%, a manuscript will be acceptable to your journal despite the meagre sample size.

Is my colleague right or wrong? Are there any minimal standards on statistics that manuscripts submitted to your journal should satisfy, and how can one find them?

A. The Nature journals do have a statistical checklist, available on Nature's website as a one-page (44KB) Word document download. We welome volunteers from suitably qualified scientists to act as statistical referees for us. (If any such people read this posting, please drop your address details in the comments.) We also welcome suggestions for improving our checklist -- are there other elements we need to add?

Posted by Maxine Clarke on December 11, 2006
Categories: Peer Review Feedback | Permalink | Comments (0)

The Council of Scientific Editors (CSE) yesterday (7 December 2006) published a guide to "promoting integrity in scientific journals". This document aims to guide editors and publishers in benchmarking their journals' policies and procedures, as well as to provide advice to scientists in their varied roles as authors, editors or peer-reviewers, in all aspects of the publication process. The comprehensive guide covers many aspects of the publication process, including workable 'conflict of interest' policies and reseach misconduct.

The full CSE guide can be seen here, at the organisation's website.
The section of the guide specific to peer-review is here.

Posted by Maxine Clarke on December 08, 2006
Categories: Ethics | Permalink | Comments (0)

Welcome to Nature Publishing Group's new-look peer-review blog. In the summer of 2006, the journal Nature hosted an online debate on peer-review. This took the form of 22 commissioned articles on a range of topics. Readers could comment on the articles on a peer-review debate blog.

We have now revamped the blog as a more general discussion forum about peer review, calling it Peer to Peer. We welcome scientists and others who have reviewed, are reviewing or would like to review for the Nature journals. This is your blog: we will post articles about the peer-review process, link to others (see Connotea section of the left-hand vertical sidebar), and answer your questions about the Nature journals' peer review policy.

Please send your brief article or question to us at the peer-reviewers' email address, and we'll feature your feedback on Peer to Peer.

The Nature peer-review debate is fully archived on Peer to Peer. Click on the category "peer review debate" on the left to see all the commissioned articles and the comments so far. We encourage you to join in the conversation with us.

Posted by Maxine Clarke on December 07, 2006
Categories: General | Permalink | Comments (0)