At the last evening keynote address for the MOHG, Jeremy Berg, the head of the National Institute of General Medical Sciences presented some early data coming out of the NIH’s new peer review policies instituted last year. As part of the new system, members of the review committees for grants are instructed to rate individual applications on a scale of 1 to 9 for the quality of five different categories: significance, investigator, innovation, approach, and environment (meaning the institutional support available to the investigator). Although these criteria are not calculated into the overall score upon which research decisions are made, Berg says they can serve as a parallel indicator of what the review panels seem to value the most. Based on 360 grants to the NIGMS as of October 2009, the criteria most predictive of getting a high overall score were, approach (.74), significance (.63), innovation (.54), investigator (.49), and environment (.37). Grant administrators, says Berg, use the data to look closer at, for example, why some highly innovative grants aren’t being awarded high overall scores and to make sure that the emphasis on approach doesn’t mean that study sections are favouring projects that look reasonable and therefore possibly ‘safe.’ It provides, Berg says, another way of looking at our portfolio.
For information on other scientific indicators and how they’re being used, check out our metrics special.