In science it pays to have friends. Peer reviewers picked by the authors of a manuscript tend to provide more favourable feedback than scientists selected by the journal’s editors. That’s the unsurprising conclusion of an analysis of more than 500 manuscripts.

The suggestion that friends are kinder than strangers in peer review has been made before; analyses of the Journal of Clinical Investigation, the Journal of Investigative Dermatology and others have concluded as much. The issue also came up last year, when a group of stem cell researchers complained that overly chummy peer review was letting in inferior papers and delaying publication of better work.

But the new paper, published in PLoS-ONE, examines a journal with an open peer review system that might be expected to discourage such biases, says Lutz Bornmann, at the Office of Research Analysis and Foresight of the Max Planck Society in Munich.

He and his colleague Hans-Dieter Daniel, at the University of Zurich, Switzerland, focused on the journal Atmospheric Chemistry and Physics. The journal, launched in 2001, uses a semi-open peer review system, publishing the vast majority of articles it receives in a journal called Atmospheric Chemistry and Physics Disucussions.

There, the manuscript is reviewed anonymously by scientists recommended by the journal editors or the paper’s authors. Anyone else interested can post a review, as well. The authors then resubmit their paper for publication in Atmospheric Chemistry and Physics. Of the manuscripts submitted between 2001 and 2006, 86% were published in both journals, while 7% only made it to the discussions journal and another 7% didn’t even make it that far.

With access to reviews and information about who wrote them, Bornmann and Daniel found some signs of bias. On papers where there was disagreement among both kinds of reviewers, those recommended by the author were more likely to provide favourable feedback and accept a paper than the editor-recommended reviewer. And, after accounting for the quality of the papers via the citations that the papers would go on to rack up, Bornmann and Daniel still found a reviewer bias – small, but still there.

It’s not hard to come up with explanations for such patterns. “The danger is really that an author suggested their best friends,” Bornmann says. Alternatively – and more charitably – the reviewers selected by authors could in a better position to know a good result in their field when they see one, compared to those selected by journal editors, he says.

If indeed there a friend bias exists, Bornmann thinks journals should try to minimize it, without limiting their pools of peer reviewers. One option would be to direct author-submitted peer reviewers to other, similar papers.

Authors might also take their own initiative. When Bornmann reviews papers, he says he ignores the author’s name and institutional affiliation and concentrates on the paper and its data. Yet he admits that he has not tested whether he, too, succumbs to any bias.

“I haven’t done my own review,” Bornmann says. “I should.”

Image: photo by nieske via Flickr under Creative Commons