The US National Institutes of Health (NIH) will continue a policy that allows grant applicants just one resubmission if their proposal is rejected the first time, the agency announced today.
The policy, launched in 2009 as part of an overhaul of peer review at the NIH, “continues to achieve the stated goals of enabling NIH to fund as much meritorious science as possible in as short a time period as possible,” Sally Rockey, the NIH deputy director for extramural research, wrote on her blog Rock Talk.
The ‘two-strikes’ policy has received negative reviews from the grantee community, many of whom argue that its effect, in an era of historically low grant-funding levels, is to randomly cull proposals of high merit.
The decision “is a characteristic refusal by administrators to admit they erred badly when they made a change that has been consistently unpopular with those it affected, in this case NIH grant applicants,” says John Moore, a senior HIV/AIDS scientist at Weill Cornell Medical College in New York, and a longtime NIH grantee.
Rockey based her decision on new data the agency has generated that continue through the end of the government’s 2011 fiscal year. As shown in Figure 2 (below) from Rockey’s blog, the average time that scientists wait to receive an award has decreased from more than 80 weeks to 56 weeks since the policy was implemented.
{credit}NIH{/credit}
As we reported last month in ‘NIH re-evaluating “two strikes” rule’:
“The NIH’s rationale for the 2009 change was that the [chance to submit an application a third time] was causing peer reviewers, either consciously or subconsciously, to favour second and third submissions over first-time proposals, creating, in effect, a queue similar to airplanes circling an airport waiting for a free runway to allow them to land.”
Today, Rockey wrote, “the ‘queuing’ of original applications and revisions has been eliminated.” In addition, she noted, a higher proportion of first-time applications are being funded compared to second-time attempts under the new rules, an outcome that was not necessarily expected. In 2011, first-time applications represented 49% of those funded, and second-time applications, 44%. (The remaining 7% were residual third attempts, left over from before the new policy was instituted.) Rockey’s Figure 1 illustrates this:
{credit}NIH{/credit}
The agency also used new data to argue that its policy is not disproportionately hurting the youngest investigators, who have less experience crafting grant applications — and fewer substantially different experiments to fall back on in designing a new application should they ‘strike out’ after two attempts. Figure 2 also shows that, under the two-strikes regime, new principal investigators waited only a couple of weeks longer to win an award than did all applicants collectively.
Rockey also addressed a suggestion that second-submission applications scoring in the top 25% be allowed a third try. Her modelling (Figure 3) showed that, if all the failed second-chance applications were funded on a third try, the agency would have been able to fund 21% fewer first-time applications and 19% fewer second attempts.
{credit}NIH{/credit}
The two-strikes policy has been the bane of many NIH-funded scientists, more than 2,300 of whom signed this 2011 letter to the NIH, pleading with the agency to return to a three-strikes system allowing a second resubmission of a failed proposal.
Rockey added that, despite holding fast to the changed policy, the agency isn’t oblivious to the pain being caused by stagnant budgets and historically low grant application success rates: “We are well aware that a lot of great science goes unfunded because paylines have decreased.”
UPDATE: Robert Benezra, the cancer biologist at Memorial Sloan-Kettering Cancer Center in New York who spearheaded the 2011 letter of protest to the NIH, had the following reaction to today’s decision:
“The central premise of our argument was that study sections cannot distinguish between an unfunded [second-try application] at the 20th percentile versus a funded [first time submission] at the 12th percentile. I have not spoken with a single study section member who feels otherwise. Therefore that [second-try] applicant should be allowed to reapply as many times as he/she likes. Otherwise the system is being culled randomly in that quality range and it would be just as fair to have a lottery.
Yes, the new system decreases the time to funding, but it does so by a random selection of meritorious applications. This is all due to the fact that in the halcyon days of doubling NIH budgets, too many people were brought into the system. The NIH should just say this and we can all be on our way.”