Peer-to-peer

From Nature 454, 928; 21 August 2008:
The US Federal Bureau of Investigation (FBI) plans to publish in peer-reviewed journals much of the scientific evidence it used to pin the 2001 anthrax attacks on microbiologist Bruce Ivins.
Ivins's suicide on 29 July means that the government's case against him will never be heard in court. The trickle of circumstantial evidence released in an investigation that had previously fingered the wrong man has lawmakers, scientists and others clamouring for more information.
In response, the FBI invited scientists and journal editors to a briefing in Washington DC on 18 August to discuss the science of the case and investigators' conclusion that a single man carried out the multiple, deadly mailings of anthrax spores. But FBI officials admit that some mysteries of the case may never be resolved. "I don't think we're ever going to put the suspicions to bed," said Vahid Majidi, assistant director of the division of weapons of mass destruction at the FBI. "There's always going to be a spore on a grassy knoll."
In lieu of expert witnesses and cross-examinations, the FBI plans to offer the evidence for peer review and will keep much of the data quiet until they are published. FBI laboratory director Chris Hassell anticipates a dozen or so papers related to the case, in addition to those that have already been published. However, Hassell says, some details of the investigation will remain confidential, so that potential bioterrorists won't know exactly what they're up against. "It's just what we have to do for national security," he says.
"Given that Ivins cannot stand trial, putting the data through the rigorous process of scientific review may be the best available alternative," says Alan Pearson, director of the biological and chemical weapons control programme at the Center for Arms Control and Non-Proliferation in Washington DC.

Continue reading "Science of anthrax case will be submitted to peer-review" »

Posted by Maxine Clarke on August 21, 2008
Categories: Ethics, Journals, Policy and Public | Permalink | Comments (0)

From Nature News in Brief (454, 564; 2008):
The system used by the US National Institutes of Health (NIH) to evaluate grant proposals does not adequately compensate for reviewer bias, affecting one in four proposals, a study finds.
Valen Johnson, a biostatistician at the University of Texas M.D. Anderson Cancer Center in Houston, evaluated reviews for nearly 19,000 grant proposals performed by around 14,000 reviewers in 2005 (V. E. Johnson Proc. Natl Acad. Sci. USA doi:10.1073/pnas.0804538105; 2008).
Each application is typically read by 2–5 reviewers, then discussed when a larger study section of about 30 reviewers meets. In the end, scores from all the study section's members — readers and nonreaders alike — are averaged together.
The system fails to account for individual bias and places undue weight on panel members who have not read the proposals, Johnson argues. He found that the top grants were largely unaffected by reader bias, but that it did affect grants closer to the funding cut-off line. Overall, accounting for reader bias changed about 25% of the funding decisions, meaning that one in four funded proposals would have been replaced by one that had not been funded.

See here for a longer version of this story, which includes an informative online readers' discussion.

Posted by Maxine Clarke on August 14, 2008
Categories: Ethics, Grants, tenure and positions | Permalink | Comments (0)

From the Editorial in the August issue of Nature Reviews Cell and Molecular Biology (9, 583; 2008):

First-time applicants [for NIH grants] compete with thousands of new and established scientists, an experience that can be intimidating and frustrating. The level of detail required for the majority of applications — for example, the exhaustive budgetary specifications — has been a burden not only for applicants, but also for reviewers. Moreover, a grant proposal can take up to 18 months to pass through the system, waiting in line behind older applications, most of which must go back and forth to applicants for amendments before approval. Of course, funding is not guaranteed: the NIH receives between 35,000 and 40,000 proposals a year, of which only 25–30% will eventually be funded.
On the basis of feedback solicited from the life-sciences community on the current peer-review system, the NIH plans to revamp the grant-review process to encourage innovative research and reward quality science. The new practice involves shortened applications that should improve the value and transparency of the review process and ensure balanced, fair reviewing across scientific fields and scientific career stages. Recruiting, training and rewarding the best reviewers is also a priority. The new process allows reviewers to focus more on the science presented and less on the details of financial requests, and compensates them for their time and effort.
Hopefully, the NIH's changes will not only foster new innovative research efforts, but they will also free up some of scientists' precious time — allowing them to spend more time planning experiments, on the bench and writing and reading scientific papers.
See also: Will NIH's overhaul be cosmetic or curative?

Posted by Maxine Clarke on August 06, 2008
Categories: Grants, tenure and positions | Permalink | Comments (0)