Peer-to-peer

Nature Neuroscience is joining a consortium of journals that enables reviews to be transferred from one journal to another, while allowing authors, referees and editors to control their degree of participation in the system flexibly. The reasons for the decision are explained in this month's (April) Editorial (Nat. Neurosci. 11, 375; 2008). Briefly, in January, a group of editors, supported by the Society for Neuroscience, implemented a system for transfer of submitted manuscripts between journals that allows voluntary participation by authors, referees and editors, known as the Neuroscience Peer Review Consortium. This consortium reduces the overall reviewing workload of the community by allowing authors to continue the initial review process when their paper moves from one consortium journal to another, once the paper has been rejected or withdrawn from the first journal. This arrangement is similar to the manuscript transfer system that has been available within the Nature family of journals , and all the other journals published by Nature Publishing Group, for almost a decade.
The neuroscience transfer system, described in detail in the Editorial and at the Neuroscience Peer Review Consortium website, is voluntary for authors and peer-reviewers. Journal editors have full discretion in deciding how to use transferred reviews. The receiving editor may choose to accept or reject a paper based on these reviews, without further consideration; to send the paper to some or all of the previous referees for evaluation of the authors' revisions; or to request a fresh set of reviews from new referees. Only comments to the authors are transferred to the receiving journal. Confidential comments to the editors are not passed along. Thus, to ensure transparency in the review process, both at Nature Neuroscience and at other journals after the paper has been transferred, journal editors encourage referees to include all their concerns about the paper in comments to the authors. According to the Nature Neuroscience Editorial, the small amount of extra time required to phrase comments diplomatically for the authors should be more than counterbalanced by the resulting improvement in the peer-review process.
Many members of the community have strong views on the issue of confidential comments, which can be found on the Action Potential blog, and which has previously been discussed at Peer to Peer. The success of the neuroscience journals' transfer system will be evaluated at the end of this year. Nature Neuroscience editors will also be evaluating the journal's participation on an ongoing basis, so they encourage authors, referees and readers to share their comments with the editors, either on Action Potential blog or privately by e-mail.

Posted by Maxine Clarke on March 31, 2008
Categories: Ethics, Quality and value, Systems | Permalink | Comments (0)

In an analysis reported in a News story in Nature this week, 222 NIH grants: 22 researchers (Nature 452, 258-259; 2008), it emerges that 200 scientists received six or more grants each from the US National Institutes of Health (NIH) in 2007. One principal investigator was awarded 32 grants, and many others got eight or nine. This is counter to the recommendation last month by the advisory panels reviewing the NIH peer-review system that researchers should devote at least 20% of their time to any project awarded a research grant (see Nature 451, 1035; 2008).
According to the Nature news story, NIH director Elias Zerhouni says that the inequities between the haves and have-nots were caused by a doubling of NIH funding between 1998 and 2003. As funding levels rose, many new PhD positions were created. Established investigators, using data produced by the new PhDs, were able to submit better grant proposals. But hordes of these grant-hungry PhDs were left standing when NIH funding flattened out after 2003. The agency now funds significantly more people over the age of 70 than under the age of 30. “We're eating our seedcorn,” says Zerhouni. Changes to the NIH peer-review system will be unveiled in mid-April.

See the related article by Gene Russo in NatureJobs (Nature 421, 381; 2008).

Posted by Maxine Clarke on March 20, 2008
Categories: Policy and Public | Permalink | Comments (0)

Noah Gray, an editor at Nature Neuroscience, asks at Action Potential blog why neuroscientists are passing on the seemingly golden opportunity to communicate with one another online, for example on published articles at a journal website, or in an online journal club. Many have expressed opinions about reasons for this reticence: Noah links to some articles in his well-argued post, and you can read other thoughts (and find links to some of the same articles) at Nature Network (for example at Gobbledygook and at Flags and Lollipops).

Here, I'd like to highlight a response by "Michael" in the comment thread to Noah's Action Potential post, as I believe this summary covers many, if not all, of the reasons why scientists in a discipline tend not to post comments on, and discuss, scientific research papers online. "Michael" writes:

I think there are a number of reasons as to why “Web 2.0” has not played much of a role in discussing neuroscience research. In no particular order:
1) Inefficiency: If I want to know why somebody used buffer x for their biochemistry experiments, or why they didn’t do control experiment xyz I email the authors, or use the phone if I know them. Why post it in the comment section of their paper, and wait for 5 weeks until they bother to check? And why does everybody else need to know about it?
2) Lack of dialogue: Commenting forums are poor venues for true dialogue. If you analyse the comment sections for the more popular entries, either here or elsewhere (for example the New York Times) there rarely is a true back-and-forth of ideas. All too often it’s 50 comments trying to be as witty as possible, with few people attempting to follow and respond to what others have written. There are rare cases where a small number of like-minded people post thoughtful comments at just the right rate to allow for a meaningful discussion. Without some type of moderation, it will remain the exception. The dynamics of a true group discussion, be it at a Journal Club or at your poster, are often taken for granted, but can’t be easily replicated online.
3) Who is commenting: It’s fine to have a democracy of opinions: in science, I don’t care so much about it. The people I want to hear/read often choose not to comment, whereas others who have nothing to say keep posting.
4) The Fear Factor: This one is obvious, and is why scientists have lab meetings or face-to-face Journal Clubs. It’s also why attempts to discuss papers online haven’t quite lived up to expectation. Ideally, we want to be honest in our opinion of a paper, but we are also human and don’t want to suffer the consequences of bruising the ego of a potential reviewer or search committee member. Staying anonymous is not the solution, since that makes it difficult for everybody else to properly evaluate the comment. After all, it does matter who is doing the criticizing.
5) Speed: Even the liveliest online discussion of a paper will drag on over hours or days. If a paper grabs my attention I will discuss at a lab meeting or Journal Club and over the course of one hour we will have thoroughly dissected it. Our attention span only lasts so long. Again, the online dynamics of who logs in at what time don’t allow for a true discourse that leads to some sort of resolution.

Posted by Maxine Clarke on March 17, 2008
Categories: Technical solutions | Permalink | Comments (1)

The newly launched journal Scholarly Research Exchange, published by Hindawi, is for original research articles in all areas of science, technology and medicine. The journal is operating a transparent peer-review system, in which authors and reviewers interact directly throughout the peer-review process. Authors submitting to Scholarly Research Exchange suggest potential reviewers, who are then approved by the journal's editors. Reviewers are asked to provide an assessment of the quality of the manuscript, a written critique for the authors, and a written commentary for the journal's readers. When the journal accepts its first articles for publication, the reviewers' commentaries and their assessment of the manuscript's quality will be published with them. In addition to these reviewers' evaluations, members of the scientific community will be able to participate in a discussion around every manuscript.
A journal with a similar, but not identical, peer-review process is Biology Direct, published by BioMed Central, currently covering the areas of genomics, bioinformatics and systems biology, immunology, and mathematical biology. Biology Direct makes the author responsible for obtaining reviewers' reports, via the journal's editorial board, operates an open peer-review process, and publishes reviewers' reports with the articles. It has been publishing articles since January 2006.

Posted by Maxine Clarke on March 12, 2008
Categories: Systems | Permalink | Comments (0)

How many papers does the typical researcher review in a year? How long do they take? And why do they do it? For the answers, read the Editorial in this month's (March) issue of Nature Nanotechnology, "Who'd be a referee?" (3, 119; 2008). The Editorial reports some of the findings of the recent Publishing Research Consortium report on peer-review (previously discussed at Peer to Peer), whose survey revealed that the "average review takes about 8.6 hours (with a median of about 5 hours) and is completed within 3–4 weeks, although there are significant differences between the four broad subject areas covered by the survey: in the physical sciences and engineering, for instance, the average (mean) is 10.4 hours, compared with 6.3 for clinical researchers. Of course, on top of this, many researchers spend considerable time reviewing grant applications for funding agencies, which can be equally onerous and possibly even more important than reviewing papers."
For similar statistics, and a more general discussion of the peer-review process in publishing, see the Nature Nanotechnology Editorial, or see here for Nature Cell Biology's take.

Posted by Maxine Clarke on March 10, 2008
Categories: Quality and value, Systems | Permalink | Comments (0)

This month's (March 2008) free-access Editorial in Nature Cell Biology (10, 247; 2008) addresses the journal's peer-review process: specifically, what the journal does to ensure that its selection process is fair.
From the Editorial:
"A legitimate question for editors at Nature Cell Biology is whether we are considering alternatives to the status quo of anonymous peer review, in particular, why we do not use a 'double blind' process (which received significant endorsement in a recent survey of the Publishing Research Consortium). The existing process, based on a thorough pre-selection by five full-time editors and subsequent external peer review by carefully selected referees, works well — individual stories of woe notwithstanding."
The editorial goes on to outline how changes to the system, when being considered, must show a demonstrable improvement to the process. Several of the commonly proposed alternatives to the 'single blind' system are discussed in this light (see here for a recent popular debate on the topic at this blog). The Editorial concludes with a summary of the process as currently run by the Nature Cell Biology editors, together with the journal's planned enhancments.
The editors welcome your views on the Editorial as comments to this post.
Further information about the Nature journals' peer-review policies are available at the authors' and reviewers' website.
Connotea tags for peer-review.

Posted by Maxine Clarke on March 06, 2008
Categories: Journals, Systems | Permalink | Comments (0)