When it’s not a trip, my day job gets in the way of my posting something to Spoonful. This week we closed the June issue of Nature Medicine, and right now I’m at the airport, about to start another ‘tour’. So, while I wait for the PA system to herd us to the plane, I thought I would blog about my day as a juror.
For the second year running, a very dear friend of mine invited me to be part of the jury for an award that her organization gives to young scientists. And for the second year running, it turned out to be a fun day out.
The award recognizes young scientists in all disciplines. As a result, the jury (composed of nearly 40 people) was a very eclectic mix of basic scientists, engineers, physicists, matemathicians, you name it. There were even editors like me, whose only expertise lies on the inexact science of rejecting papers.
The discussion, which took the best part of the day to go over something like 40 finalists, was free-form. One of us would go over the candidate, and the rest would ask questions or bring up caveats about each of them, trying to understand the importance of their contributions.
I must confess that, halfway through the session, I started feeling sorry for those scholars who have to decide on, say, people’s grants.
In the case of our jury, we found it was pretty tricky to decide how much weight to give to the candidates’ letters of recommendation, to their number of papers and the journals in which they were published (someone referred to the high-profile journals as ‘vanity journals’, which struck a chord with me), to the number of citations, and to a plethora of factors that, one way or another, represent the blood, sweat and tears of a scientist.
Does a scientist who has three patents in the past five years, but only three papers, each of which had been cited just three times deserve more recognition than the scientist with five Nature papers and 1000 citations? Does a scientist who works in a hot field and has made nice contributions deserve more credit than another one who works in a less glamorous, lonelier field and has made equally profound contributions?
These questions aren’t always easy to answer, but my friend got it right because she quickly realized that there’s strength in numbers. So, if you have 40 judges from different fields and with different ways of evaluating science, chances are that the truly outstanding pieces of work will triumph over the rest. I’m therefore confident that our eclectic group made the right choices.
But returning to those grant evaluators, I was saying that I felt sorry for them because, if we managed to screw up and made a mistake yesterday, it won’t make much of a difference for the scientists who should have won and didn’t despite our best efforts. They’ll carry on and will still be great scientists. In the case of grant decisions, though, getting them wrong can lead to a lab’s shutdown, to fired postdocs and to truncated careers, which, alas, are becoming more and more common.
Photo by JasonUnbound