Peer-to-peer

A news story in Nature Medicine (14, 1133; 2008) discusses an investigation into the publication status of the clinical-trials literature (PLoS Med., doi: 10.1371/journal.pmed.0050191; 2008), which concludes that positive results of clinical trials for drugs or devices have a higher chance of getting published than negative trials. The study further concludes that when the trial sponsors publish the results from 'pivotal' trials only 76% of the time.
Studies reporting a statistically significant difference were more than three times as likely to be published. This 'positive publication bias' is a serious problem, because it can make a drug or device appear in the literature to be more effective than it is. Ida Sim, a co-author of the study, told Nature Medicine: "We have this idea of practicing evidence-based medicine, which is predicated on having a full and complete evidence base. But when the evidence base is skewed, we can't really do this."
According to the Nature Medicine article, a paper in Science (319, 1340–1342; 2008) indicates that "the FDA Amendments Act of 2007 has improved transparency, because the law mandates that sponsors or primary investigators of clinical trials for approved drugs post a summary of their results in a national open-access database. The lead author of the report, Deborah Zarin, oversees the ClinicalTrials.gov registry at the National Library of Medicine of the US National Institutes of Health and is in charge of ensuring the results are posted in compliance with what the new law. According to Zarin, "for the trials that are covered by this law, the results database should have a big impact on disseminating medical knowledge, because the results have to be publicly available."
But not every type of clinical trial is covered by the legislation, nor does it directly affect medical journals. Although Sim applauds the FDA Amendments Act of 2007, she adds that it "doesn't address the issue of not publishing trials in medical journals. They remain one of the most influential and biased sources of information." "

Posted by Maxine Clarke on November 27, 2008
Categories: Quality and value | Permalink | Comments (0)

Over the past four years, the editors of Nature Chemical Biology have enjoyed getting to know chemical biologists from around the world and hearing about their diverse research. In the journal's December Editorial (Nat. Chem. Bio. 4, 715; 2008), the editors describe how they select papers. From the Editorial:

The review process at Nature Chemical Biology includes two steps: editorial assessment and external peer review. We first read and discuss each manuscript in light of our editorial criteria and determine whether the manuscript should be sent for external peer review. For manuscripts that are not initially selected for external review, we contact the authors to decline the paper, and in some cases we encourage resubmission of a revised manuscript. In the latter case, we send the authors a decision letter that contains a web link for submitting a revised paper and encourage them to discuss the study further with the editor. Papers that are selected for external peer review are sent to a panel of scientific referees covering appropriate chemical and biological expertise, who are asked to assess the technical merits and scientific advance of the paper. After review, the editors discuss the manuscript in the context of the referee reports and make a decision to accept or decline the work, or to request a revised manuscript. Most published manuscripts have undergone more than one round of review by the same referees to ensure that all technical concerns have been addressed.
How do we determine which manuscripts are sent for external review? Our first consideration is whether the submitted manuscript falls within the scope of the journal. In principle, chemical biology papers should have a balance of chemical and biological components. However, for most current manuscripts the major advance is focused in one discipline. As a result we consider papers across this spectrum, ranging from fundamental chemical studies that are applied to understanding biological systems to major biological advances that were enabled by chemical approaches. Second, we consider whether the paper reports significant conceptual or methodological advances that are likely to open up new avenues of research in the field. In assessing a manuscript's novelty, we rely on our editorial expertise and familiarity with the field's criteria, but we also closely examine the published literature to provide context for the reported discoveries. Finally, because the journal's audience includes both chemists and biologists, we favor papers that are likely to appeal not only to specialists but also to our broad readership.

The full Editorial is here.
Peer-review policies of the Nature journals.
More about peer-review at the Nature journals.

Posted by Maxine Clarke on November 20, 2008
Categories: Journals | Permalink | Comments (2)

The US National Institutes of Health (NIH) will introduce new guidelines in January 2009 allowing biomedical researchers to amend and resubmit a failed funding application only once, as part of an overhaul of the peer-review system for evaluating grant proosals (see Nature 455, 841; 2008). Applicants whose grants are unfunded after the second submission may reapply only after designing a new proposal. NIH has previously suggested not allowing resubmissions, but decided against this step after an outcry from researchers (see Nature 453, 835; 2008).
NIH estimates that the move will reduce the number of applications by up to 5,000 — welcome news as it struggles to evaluate about 55,000 applications this year. In 2007, only about 30% of awards were granted to first-time submissions.
According to a comment to the Nature News story by Jeremy Green, in the current UK funding system, the research councils and the Wellcome Trust operate a no re-submission policy, although depending on the subject area an applicant might be able to submit a proposal rejected by one funder to the others. However, applicants to the research councils do get an opportunity before the grant review panel sits to see and respond to the reviewers' comments.

Posted by Maxine Clarke on November 14, 2008
Categories: Systems | Permalink | Comments (0)