A couple of days ago we got a comment from one of our referees, saying that it’s unreasonable for us to get more than three reviewers for any given paper that we consider for publication. He stated that having to deal with the comments from more than three referees places an undue burden on authors, which may put the community off from sending us new submissions.

We are certainly familiar with some (unfavorable) comparisons that are made between our journal and other publications that use only two reviewers per paper. I’d like to say, first, that we don’t get more than three referees for any given paper as a matter of course. When we do, it’s sometimes because one of the referees has not reviewed for our journal before. We aren’t therefore sure how his/her comments will compare with what other, more experienced, referees may have to say about the paper — sometimes new referees are either too tough or what we call “wet”. So, as different journals have different standards and criteria, referees often go through a “training period” during which they come to be familiar with the kind of papers a journal looks for.

Second, a good number of submissions to Nature Medicine tend to be multidisciplinary, making it very difiicult for just two people to evaluate the full manuscript. For starters, most of our papers include human and animal data. So, right off the bat we may need one person with technical expertise on the animal experiments and another person to advise us on the potential relevance of the findings to human disease. If you then consider that you may want to have a second opinion about the same points, we’re already talking about at least three reviewers and have not even started talking about papers that stretch over two or more disciplines.

Third, when we look at all of the referees’ concerns, we don’t necessarily ask authors to address every point, particularly if the criticisms bring up points that are clearly part of a subsequent study. So, two referees times two does not necessarily equal four sets of comments.

To finish, I should say that two or three years ago we followed up with people who had published in our journal, some of whom had to go through the “four-referee ordeal”. We ask these authors if they felt that the review process had significantly improved their paper. The overwhelming majority of these authors agreed that the referees’ criticisms had really made a difference and were frankly appreciative of our peer-review process. I guess you can’t please everybody all of the time.