Peer-to-peer

The Journal of Biology (8, 1; 2009) has announced an experimental policy of allowing authors of submitted manuscripts to opt-out of re-revew on occasions where the peer-reviewers require revisions, including the addition of data. In these cases, the journal will not publish the referees' reports with the manuscript, but instead will publish an accompanying commentary.
This adds a new model of peer-review to those previously described in Nature's peer-review debate of 2006. Journals using unusual forms of peer-review system include Biology Direct, Atmospheric Chemistry and Physics, Signaling Gateway (a database publication), Electronic Transactions on Artificial Intelligence, and the BioMed Central (BMC) journals - BMC publishes Journal of Biology. Further discussion on systems of peer-review since 2006 is archived in a series of posts on this blog.
It will be interesting to see how the Journal of Biology experiment is received by readers and peer-reviewers (no doubt that it will be popular with authors). As well as the question of the accuracy of pubished papers whose authors have not addressed technical criticisms, there is also the question of motivation of peer-reviewers to write detailed reports, if they know their advice can, if the author wishes, go unheeded. Particularly strange to me is the decision by the journal not to publish the referees' reports with the unrevised manuscript, but instead to publish an independent commentary. Will this commentary always accompany publication of incomplete manuscripts, or might it be delayed? Will it be written by one of the peer-reviewers, or if not, by someone who has access to the reports?

The Nature journals' peer-review policy and procedures are described here. This web page contains an archive of free-to-access editorials in many Nature journals that discuss aspects of the peer-review system and process.

Posted by Maxine Clarke on January 29, 2009
Categories: Systems | Permalink | Comments (4)

"Scientific publishing depends on expert peer reviewers. Instead of perpetually arguing about the reliability and fairness of peer review, authors, editors and referees should seek to optimize this time-tested system." So opens the January editorial of Nature Neuroscience (12, 1; 2009).
The editorial discusses the media reporting of a ferocious argument about the merits of a paper published in Cell , and a subsequent blog debate hosted by The Scientist. The controversy between scientists in this discipline concerning this paper "has again ignited a debate on the flaws of editor-managed anonymous peer review", write the Nature Neuroscience editors. "We maintain, however, that despite occasional unfortunate lapses, anonymous peer review remains the best quality-control process that we have." The editorial goes on to discuss how journals can best optimize the process.
In Nature News this week (457, 245; 15 January 2009) another technical dispute is discussed, this time concerning a widely circulating preprint attacking much of the published research in social neuroscience involving functional magnetic resonance imaging (fMRI). This affair is exacerbated by the fact that the circulating preprint is not due to be published, with responses from the authors of some of the criticized studies, for another nine months, according to the Nature News story. The criticisms have already been covered in publications such as Newsweek, as well as the inevitable blogosphere outpourings - and at least some of the criticized authors say the first they heard of the preprint was when they were contacted by journalists.
How scientific reports should be peer-reviewed and, after publication, scrutinized are matters that are decided upon by the journals, their editors, and their publishers and/or societies - in the case of articles submitted to and published in the Nature journals, the peer-review process is described here, and the post-publication corrections process here. Good journals have processes for investigating technical criticisms and complaints about the papers they publish. Nature's, for example, is here. Often, a resolution is not clear-cut at the outset, when the complaint is first received by the journal, however clear it may be in the mind of the complainer. A proper outcome depends on independent peer-reviewers, as well as editors, examining the complaint together with a measured response from the study's authors. Playing out such investigations in the kangaroo courts of the popular press, or in unfettered comments on the Internet between people who have been described as "recreationally outraged", not only obscures logical, technically informed investigation, but unnecessarily exacerbates emotions and arguments so that, in the end, all that is remembered is the heat - not any light.

Posted by Maxine Clarke on January 16, 2009
Categories: Ethics, Quality and value, Systems | Permalink | Comments (2)

Frank Gannon imagines a modern response to Darwin's research grant application (EMBO Reports 10, 1; 2009):
"We were a bit puzzled by your handwritten application for funding as it shows a staggering disregard for our practices and rules relating to requests for research support. We usually receive applications by electronic submission, and they must be received before 5 pm on the closing date. However, we realize that our current instructions do not formally preclude non-electronic communication and therefore we had little choice but to consider your application.
We sent your proposal to a panel of international expert referees who have not worked or published with you during the past 10 years. Their comments are summarized below and we hope that you will find them helpful should you choose to resubmit in the future. Although your project departs from the routine—a refreshing change from modern research projects that tend to be merely incremental steps with no real impact on the universe of relevant knowledge—I am afraid that your application for funding has been rejected."
Read on for the details of the three reviewers' reports, and why the potential funder concludes: "There are unusual and therefore intriguing aspects to your idea and the scale of your ambition is impressive. In the final analysis, however, our decision is based on a simple test of whether the work will have a major scientific impact—in other words, will people consider it to be an important piece of knowledge in a hundred years' time? Regretfully, we do not think that your proposed work will yield sufficient insight to meet this requirement."

Nature's special set of articles to celebrate Darwin's achievements is available here.

Posted by Maxine Clarke on January 09, 2009
Categories: Grants, tenure and positions | Permalink | Comments (0)

Via press release, The EMBO Journal will be publishing online author and referee comments from this year (2009). “The EMBO Journal has been our flagship publication for 27 years, sharing knowledge broadly within the molecular life sciences community,” said Hermann Bujard, director of EMBO (European Molecular Biology Organisation). “We are excited by the editorial changes that will make publication of research findings more transparent, complete and visible.”
By making the evaluation of manuscripts visible to everyone, The EMBO Journal aims to encourage constructive referee and author argumentation. Younger scientists will gain valuable insight into how to publish their research findings as well as how to deal with criticism.
"The EMBO Journal has an efficient and reasonable editorial process,” writes Executive Editor Pernille Rørth in an advanced online publication of the editorial in the first issue for 2009 (EMBO J. 28 , published online 4 December 2008). “A transparent editorial process will help demystify decisions.”
Beginning with manuscripts submitted in 2009, a supplementary process file will be included with the online publication of papers. This file will show all dates relevant to manuscript processing and communications between the author, editors, referees and comments to the decision letter. Readers will learn about why referees find the paper interesting, any gaps they may have identified in the initial research findings, and how the gaps were resolved in revision. Referee identities will be anonymous and confidential comments between the referee and editors will remain so. Authors will have the option to decline publication of the editorial process when they submit manuscripts, but are encouraged to participate.
Instructions for authors are available here.

Posted by Maxine Clarke on January 05, 2009
Categories: Journals, Systems | Permalink | Comments (1)