Talk about peer review at SfN
One important thing I get out of this meeting is the opportunity to talk with scientists about how the journal is doing and what we might improve. People seem generally happy with the papers that we’re publishing, but I’ve gotten a lot of interesting suggestions too.
The most controversial one was that we should publish the reviews of accepted papers (anonymously) online. Some people loved this idea, saying that reviews contain a wealth of information, from the reasons that the referees found the work exciting to the potential limitations of the authors’ approach, which would be especially helpful to scientists reading papers outside their field. Another potential advantage would be to make the editorial decision process more transparent to readers. Knowing that the reviews would be posted might also influence referees to write them more carefully and do a better job of citing references to document their concerns about novelty or conceptual advance.
The naysayers are equally passionate, telling me that posting reviews would place an additional burden on the referees, making them more reluctant to review and probably leading them to express their opinions less candidly. People worry that allowing a large number of readers to see the reviews might increase the odds of someone identifying a referee. One scientist said that editors would be dodging their responsibilities by posting the reviews online, making it look like we were trying to shift the blame for the decision onto the referees. Others note that many flaws identified during the review process will have been fixed by the time the paper is published, so the criticisms raised by referees are no longer relevant.
I’m sympathetic to these concerns. Of course we would never post reviews without the permission of the affected authors and referees, and I don’t advocate making it a regular habit for all accepted papers. On the other hand, I can imagine that it would be helpful to readers in certain cases. It will be a while before we decide whether to try it, but in the meantime, I’d be interested in your opinions on the subject.
