The US National Institutes of Health (NIH) is considering reversing a highly unpopular policy that allows its grant applicants only one additional try at winning funding if their application is at first rejected.
Senior leaders at the US$31-billion biomedical agency in Bethesda, Maryland, will decide in the next several weeks whether to abandon a ‘two strikes and you’re out’ policy that was instituted in January 2009 as part of an extensive overhaul of peer review at the agency. Before then, grant-seekers were allowed a third try if a proposal twice failed to pass muster with peer reviewers.
A spokeswoman for the NIH said that the agency cannot comment on whether and when it will return to a three-strikes policy, because the decision is still under active consideration by senior officials.
“It’s about time” that the biomedical agency reconsidered, says Robert Benezra, a cancer biologist at Memorial Sloan-Kettering Cancer Center in New York, who spearheaded a 2011 letter from more than 2,300 scientists to the NIH, asking it to return to the three-strikes system.
The NIH’s rationale for the 2009 change was that the three-strikes rule was causing peer reviewers, either consciously or subconsciously, to favour second and third submissions over first-time proposals, creating, in effect, a queue similar to airplanes circling an airport waiting for a free runway to allow them to land. “Support for meritorious science may be delayed if initial submissions are placed at the end of the queue,” the expert group that reviewed peer review for the NIH concluded in a draft report in 2008 that showed the success rates for first-time applications falling from more than 60% in 1998 to 30% in 2007. (See page 33 of the report for the graph plotting these numbers.)
However, the change generated tremendous push-back from scientists, who have complained of it nearly incessantly to the NIH Office of Extramural Research. But after the agency received the petition from Benezra and his co-signers, it held firm to the two-strikes policy.
Sally Rockey, NIH deputy director for extramural research, published the NIH’s response to Benezra in a blog post, which included a graph that showed first-strike applications for the NIH’s mainstay ‘R01’ grants climbing between 2008 and 2010 as a percentage of all funded R01 grants. The new policy “certainly has achieved the intended goals: the number of applications funded [on the first try] is increasing and there is no queue piling up at the [second try] level,” Rockey wrote.
The critics retorted that, in the absence of third strikes, of course more first-strike applications would be funded; they also noted, ironically, that 100% of successful first-strike applications would be funded if the second strike opportunity were also eliminated.
Benezra and his allies argue that it is impossible for peer reviewers to discriminate between a proposal that scores in the top 10% of applicants and one that scores, say, at 19%. They say that the peer review process, by its nature, is not sufficiently fine-tuned to discriminate at that level.
Thus, they say, the upshot of the two-strikes policy, in an era of stagnant NIH funding and historically low grant-application success rates, is that highly meritorious proposals are randomly culled, and their authors sent back to the drawing board. There, they must craft applications so substantially new that months and years of work in an area often needs to be jettisoned. For younger investigators, the situation is particularly problematic, the critics argue, because less-experienced applicants are on a learning curve in writing grant applications, and because they do not have a body of work to fall back on should a particular proposal be rejected twice.
Some younger investigators say that they are feeling the pressure. “It was really grueling knowing you have to be almost perfect — or as perfect as can be — because you only have one more chance,” says Jill Locke, a 28-year-old postdoctoral student at the University of Pennsylvania in Philadelphia, who just submitted her first NIH grant application: a K01 career development award. (The two-strikes rule applies to career development awards as well as to R01s and many other grant types.)
“I would be very happy if they would reinstate” the third strike, says one 40-year-old cancer biologist at Memorial Sloan-Kettering Cancer Center, who failed last year to win her first R01 grant after two submissions, and who recently submitted a new R01 application. Failing after her second attempt was “just really bad,” she says. “It’s just better if you have one more chance.”
Even the most seasoned investigators have their share of heartburn with the current process, says John Moore, a senior HIV/AIDS scientist at Weill Cornell Medical College in New York. At the National Institute of Allergy and Infectious Diseases, where Moore competes for grants, the current payline for applications from established investigators is 6%.
“I defy any group of reviewers to distinguish in any meaningful way between a 6% and a 12% percentile grant,” says Moore. “If a reviewer doesn’t like how you use semicolons, or is having a bad hair day before the coffee kicks in, or was once threatened by one of your papers, you can fall from 6% to 12% in a flash, without any reflection on the actual merits of the application. That randomness has always been in the system, but its impact is magnified by such a low [success rate]. So, having a third chance literally doubles the chance an applicant can overcome a randomly (slightly) negative comment or two that was sufficient to take the application out of the funding range and into the also-rans.”
CORRECTION: An earlier version of this blog used the term ‘success rate’ in place of ‘payline.’ The terms are related, but not the same. A payline is the percentage of investigator-initiated grant applications being funded by an NIH institute at any point in time. A success rate, by contrast, is the percentage of grants funded by an institute during an entire fiscal year. For example, the success rate for National Institute of Allergy and Infectious Diseasesgrant applications in 2011, the most recent year for which data were available, was 20.2%.