Peer-to-peer

Ariberto Fassati of the Division of Infection and Immunity, University College London Medical School, writes in Correspondence in this week's Nature (447, 528; 2007):

Sydney Brenner and Richard Robert's request in Correspondence (Nature 446, 725; 2007) for authors to conserve records of their work and make them freely accessible is of great importance to historians of science.

However, unlike an artist's preparatory sketches or a novelist's drafts, scientific papers describing major discoveries have gone through the process of peer review. Reviewers often make significant contributions in shaping discoveries. They suggest new experiments, propose novel interpretations and reject some papers outright. Clearly, this is also important 'behind the scenes' work by scientists usually at the forefront of their discipline, and is an intrinsic part of the scientific process. It is well worth keeping a record of such work, for no history of science will be complete and accurate without it.

I therefore propose that journals' records should be made publicly available after an adequate lapse of time, including the names of reviewers and the confidential comments exchanged between editors and reviewers. The Nobel Foundation makes all its records available after 50 years, as do many governmental and other institutions. This delay may be reduced for scientific journals to, perhaps, 15 or 20 years. This is also likely to have a positive impact on the peer-review process itself.

The scientific community and future historians will gain from this transparency and from full knowledge of all the events that have contributed to a great discovery.

Posted by Maxine Clarke on May 31, 2007
Categories: Policy and Public | Permalink | Comments (0)

Nature’s Journal Club

Our latest blog (link above) is the blog for the Journal Club, a weekly column published in Nature’s Research Highlights pages. Each column presents a researcher’s choice of reecnt paper, explaining the reason why he or she is enthused about it. At the Journal Club blog we invite readers to discuss the subjects raised in the columns. Please do take a look at the entries on the blog, and, as my colleague Oliver Morton puts it, "enrich their comment threads with your insight and speculation."

(Cross-posted on Nautlius, the author blog.)

Posted by Maxine Clarke on May 29, 2007
Categories: Technical solutions | Permalink | Comments (0)

Reviewing appeals : Article : Nature Immunology

This month's Nature Immunology editorial (8, 541; 2007 -- link above); which is freely available, describes the journal's peer-review process and how it deals with appeals against decisions not to publish a submitted paper. This process is similar in all the Nature journals that publish original scientific research. From the editorial:

We believe that constructive critiques made during the review process improve manuscript quality, whether the manuscript is ultimately published in Nature Immunology or in another journal. Clarification of the review and appeal processes, as provided here, should assist authors in their preparation of manuscripts. We hope that by offering this advice to authors we can lessen frustration should a negative decision be rendered.

The full text of the editorial can be accessed via the link at the top of this post. Comments are welcome.

Posted by Maxine Clarke on May 25, 2007
Categories: Systems | Permalink | Comments (1)

C. Cristofre Martin and Kenneth B. Storey write:
Scientists are often involved in the peer review of grant applications and/or manuscripts submitted for publication. We rely on the anonymity of the system to allow us to be completely frank and unbiased in the comments that we provide to the author(s) of the article or grant.
However, we wish to point out an alarming situation. The now heavy reliance on electronic means of communicating between reviewers and publishers/granting agencies in the form of MS Word and other electronic documents, generated PDFs, and other user-generated file formats has created in a situation where anonymity can no longer be ensured.
The reason for this breach in security is that most state-of-the-art software applications will embed information about the creator of the document with the normally invisible metadata of the file. This metadata can be viewed by means as simple as opening the file within a text editor application or by viewing the creator information for a file within the operating system such as 'Get Info' in the Mac world or 'Properties' on PCs. Typically, the source of this metadata is the user account information that is associated with the specific computer being used to generate the document.
Authors, journal editors, publishers and granting agencies need to be cautious about how 'anonymous' information is transmitted between the creator and the recipient. Creators of anonymous documents should check that the programs that are used to create their documents are secure and if not, adjust security settings where possible or delete creator information in the file properties before sending off their reviews. Publishers and granting agencies should also consider adjusting dissemination methods such that original reviewer-created documents are never forwarded directly to authors. The greater use of Web forms for both the input and transmission of reviews is one obvious solution.

C. Cristofre Martin, Department of Biochemistry, St. George's University, St. George's, Grenada , West Indies.
Kenneth B. Storey, Department of Biology, Carleton University , Ottawa , Canada.

[Note from Maxine: For information, Nature Publishing Group journals use a Web-based peer-review system to ensure anonymity, as do many, but not all, other publishers. The Nature journals also require large datasets and other supplementary information to be deposited by sumbitting authors into a public database or supplied on CD/DVD for the purposes of peer-review. We do not allow authors to post such supplementary information solely on their own or their institutional websites, partly for the reasons outlined by Drs Martin and Storey. Further details of our policies can be found at the NPG authors and reviewers' website]

Posted by Maxine Clarke on May 22, 2007
Categories: Technical solutions | Permalink | Comments (0)

Respectful Insolence: Cranks against peer review

The post linked to here is a long article about some recent "anti science" or "crank" attacks on the scientific peer-review system. It is a stimulating and informal read, concluding that the peer review system may have its faults, but that it is sensible to be wary of outspoken criticisims of it.

Posted by Maxine Clarke on May 18, 2007
Categories: Quality and value | Permalink | Comments (0)

Peer Review: A Necessary But Not Sufficient Condition

This is a link to an interesting post on the Real Climate blog about how peer-review aims to weed out poor science, whether it is successful at that goal. The examples given are from the field of climate research, and provide a telling account of how accuracy can be lost, advertently or inadvertently, in the glare of publicity. The post concludes: "even when it initially breaks down, the process of peer-review does usually work in the end. But sometimes it can take a while. Observers would thus be well advised to be extremely skeptical of any claims in the media or elsewhere of some new "bombshell" or "revolution" that has not yet been fully vetted by the scientific community."

Posted by Maxine Clarke on May 17, 2007
Categories: Quality and value | Permalink | Comments (0)

Notes from the biomass: Warnock’s dilemma is a blog post on the discussion about lack of comments on scientific papers that are opened up for reader feedback online. Mr Warnock has a Wikipedia entry (which can be found via the Notes from the Biomass link) explaining why he thinks lack of comments does not (necessarily) indicate lack of interest.

Posted by Maxine Clarke on May 17, 2007
Categories: Systems | Permalink | Comments (0)

Adventures in Ethics and Science: Why wouldn't this be a good way to do peer review?

At the link above is discussion on the Adventures of Ethics and Science blog about the possible benefit of a peer-review system in which the peer-reviewers are scientists paid to reproduce the experiments being reported in the submitted study. Janet D. Stemwedel asks:

Aside from the concern that the journals would need to find money to create and support (with lab space, materials, etc.) these positions -- which might well raise the price of journals beyond where they are already (or raise operating costs for open access journals) -- are there obvious reasons that a plan like this would be a bad idea?

Posted by Maxine Clarke on May 15, 2007
Categories: Ethics | Permalink | Comments (0)