Peer-to-Peer

Perspective: The pros and cons of open peer review

Thomas DeCoursey

Should authors be told who their reviewers are?

The goal of any change in the peer review system must be to improve the quality of review, where quality is determined by two distinct functions: filtering manuscripts for publication in a given journal; and making constructive suggestions on how the manuscript or study could be improved. Would open review (in which reviewers sign their reviews) accomplish this goal? I have experienced several cases of open review, intentional and unintentional, with mixed results.


The main disadvantage of open review is the likely accumulation of ‘enemies’ who may later try to torpedo one’s own manuscripts or grant applications. And this occurs, it seems, regardless of the outcome of peer review. One author, for example, expressed his extreme displeasure at my review of his manuscript, despite its being eventually accepted for publication (with my support). I had made one error in around ten pages of critique. That authors ought to be better informed on the subject they have recently been studying in depth than reviewers evidently was not obvious to this author.

On the other hand, anonymous review does not always stop authors harbouring resentment if their paper is rejected. In a small field, there may be only a handful of genuine experts who have sufficient experience with the techniques, pitfalls and potential artefacts of the system, and authors are often likely to know who the reviewer is. Even if it is not obvious, most authors try to guess the identity of their reviewers from their comments or recommendations. For example, one author angrily accused me of rejecting a manuscript that I had not in fact reviewed. The author assumed it was me because the offending reviewer had recommended citing several of my papers.

But I do think that there are several advantages to an open peer-review system. First, reviewers would be more tactful and constructive. I admit that I have used sarcasm when reviewing studies that seem to be thrown together haphazardly. Sometimes I feel that I put more thought into my review than the authors have in designing the study and writing the manuscript.

Second, reviewers with a vested interest in suppressing the publication of a manuscript could be more easily unmasked by authors. Although manuscripts are rarely reviewed by a single reviewer, anonymous review does offer unscrupulous reviewers more opportunities for blocking publication without repercussion.

Third, a completely open review system would have reviewers’ names published in a footnote to each paper to further encourage reviewers to do a thorough job. When bad science is published, the negligence of reviewers can be as aggravating as the incompetence of authors.

Regardless of the type of peer review used, there is a clear need for an editor to mediate the exchange and assume responsibility for the final decision. Reviewers can give their expert opinion, which might be honest, tainted by emotion, or even an overt attempt to suppress the manuscript. The authors can rebut these arguments. But it is the editors who must determine whether the reviewer has noble motives.

Although I accept that open peer review can provide value, I doubt it can be implemented in full in our modern competitive world. A half-way house would be more realistic, in which reviewers make open constructive suggestions for revision or additional work only after the manuscript is provisionally accepted for publication. This is similar to the member/editor system used by the Proceedings of the National Academy of Sciences. The member reviewing a manuscript is identified to the authors only if and when the paper is accepted for publication. For rejected manuscripts, blind review allows the reviewers to remain hidden in anonymity and the editors to hide behind the reviewers’ judgment in their decision to reject. Maybe this is for the best. Publishing reviewers’ names alongside accepted papers still seems worthwhile. The responsibility for publication is then shared among the authors, the journal and the reviewers.

Tom DeCoursey is a professor of molecular biophysics and physiology at Rush University Medical Center in Chicago. He has studied ion channels since the late 1970s and is particularly interested in voltage-gated proton channels.

Read more See this article in Nature’s web focus here

Comments

  1. Report this comment

    Dan Kolker said:

    The real question is: should reviewers be told who the authors are? We have all seen, in our various fields, papers by prominent scientists accepted at top-name journals, even when deep inside we have felt that the quality of the work alone probably does not merit such prominence. In some cases a reviewer may feel compelled to ask fewer questions of a prominent researcher than he or she might of a more junior scientist.

    Other fields, for example philosophy, regularly require that journal submissions be anonymous. Not only do the authors’ names only appear on a detachable cover sheet, but the manuscript itself must be devoid of identifying text, such as “We previously showed” or “earlier published results from our group indicate”. This frees the reviewer from bias, overt or unconscious, as to whether the paper should be accepted, revised, or rejected.

    Nature could become a leader in the field by requiring authors to remove identifying information from their manuscripts.

  2. Report this comment

    Dr.Dinesh Kumar Singh said:

    I fully agree with the style of PNAS where the name of reviewer is revelaed only if the paper is published and not otherwise.It is also a wise decision to put the name of the reviewer at the footnote of the published paper so that the readers may really know at the end of the review process whether full justice was done in the reviewing process. Another thing which will surely help in an impartial review process is to start the process as “both ways blind”, meaning even the referees should not know where from or whose paper is being asked for review.Some people just recommend the names of their friends for reviewing their paper, while they had already discussed the results or have “secret” collaborations.Having staring the review process both ways blind folded and make it both ways open when the paper is published would do more justice than the current process.Thank you.

  3. Report this comment

    Eric Perlman said:

    I can see the merits of the arguments brought up by Drs. Clarke, Kolker and Singh on this issue. I think one additional issue needs to be brought up, and that is that from the author’s point of view, sometimes it is useful to know the reviewer’s identity in order to help gauge the pertinence and correctness of a comment. This is not only to help with hidden agendas, as Dr. Clarke correctly points out, but also to help with true scientific issues. I have specifically asked journals to reveal my name to authors as the reviewer when I have major comments on a paper that refer to an aspect where I feel I am an expert. I feel that by doing so the authors are forced to take notice and realize the importance of this remark. Once in the past when I did not do this the author was able to argue around the point, so I feel this can be helpful, although it does not necessarily require additional provisions by existing journals.

  4. Report this comment

    Respectful Insolence said:

    Open peer review: an idea whose time has come?

    Over at the Nature blogs, they’re soliciting comments and opinions about open peer review: The goal of any change in the peer review system must be to improve the quality of review, where quality is determined by two distinct functions:…

  5. Report this comment

    Joshua Sharp said:

    I am in full agreement with Drs. Kolker and Singh, in that I think that “double-blinded” reviews serve the community best by prompting the reviewers (and the authors) to focus on the science and the critiques, rather than the reputations of those involved. While I appreciate the point of Dr. Perlman, I think that the argument of the reviewer should be able to stand upon its own merit; the editors will still have the identities of the reviewers to consider when weighing their concerns, and they may choose to emphasize certain points to the authors if they feel a reviewer’s expertise warrants especial consideration.

  6. Report this comment

    Dr. Haitham Idriss said:

    I think there are merits to both arguments. However, I am still in favour of keeping the names of peer-reviewers anonymous to authors, which I believe would add more objectivity to the peer-rview process. However, we adopted a middle approach at Annals of Alquds Medicine. We do not reveal the names of reviewers to authors, but we publish the names of all reviewers who peer-reviewed for us in each issue of the journal, without stating which article they reviewed and whether it was accepted or not.

    Dr. Haitham Idriss

    Founder and Editor-in-chief

    Annals of Alquds Medicine

    http://www.najah.edu (select AQM icon).

  7. Report this comment

    Rajendar K. Singal said:

    Journals must adopt an open review system, so that the names of the referees are revealed to the author, and the author should have the right to ask questions from the reviewers and editors if his/her papers are declined fror publication.

    Important scientific associations should request a vote from their members on the open versus the present blind referee system. Blind review favours the experts and established researchers, and stifles innovation. Relatively unknown authors have very little chance of publishing their ideas and experimental results, unless the ideas are supportive of the expert’s viewpoint.

    A major advantage of the open review process is that cheating in experimental results and/or stealing of another person’s ideas without giving proper credit to the original ideas can be drastically curbed.

    Rajendar K. Singal

    9084 Alpine Peaks Avenue

    Las Vegas, NV 89147.

  8. Report this comment

    Eric Perlman said:

    In an ideal world Dr. Sharp’s point of view would prevail. But ours is not an ideal world, even in science.

    Twice in the past five years I have been asked by an editor to serve as a referee for the referees. What I mean by this is that a paper had received a negative report, with which the authors had disagreed, and perhaps a second (in one case) from a second reviewer. In one of the two cases this had gone two rather than one round.

    But in each of these cases, the author and the referees had been talking past one another, holding to very hard and fast points of view that held very little room for the other. In one of these cases, it turned out that the identity of the referee was reasonably clear from the arguments — but this was only the case because it was a field that I was intimately familiar with, as one of the ten or so other primary actors!

    As Dr. Sharp surely knows, this is not always the case, and in my own experience as an author I have often found cause to disagree with a referee — sometimes rightly, sometimes not. In the latter cases it would have been helpful to know who the referee was because this might give me a hint as to what s/he was thinking.

  9. Report this comment

    Atul Tiwari said:

    I can see a lot of debate going on the pros and cons of open peer review. I personally feel and have also experienced that it is always better to have a blind folded system for peer review. The biggest advantage lies in its authenticity and credibility. The authors would know that their manuscript rejection is not due to differences in institute, region, gender, creed, race or other similar factors.

  10. Report this comment

    Dr. Stephen Cose said:

    Prof. DeCoursey’s article highlights a very real, if largely ignored problem, of professional misconduct in the review process. In the race to publish the latest and greatest results, a rival lab may be tempted to hold up a review in order to publish at the same time. Every one of us that has a paper rejected doubtless feels harshly done by, and the question then becomes one of whether one felt that their manuscript was reviewed fairly. As many have already pointed out, the current system may not be the fairest, and this, after all, is the primary goal in order to satisfy Author, Reviewer and Editor. It seems that the fairest way to review is double blind – thus freeing the reviewer to concentrate on the science, rather than ingratiate themselves with the Corresponding Author during the review process. Allowing Authors to identify the reviewer, and vice versa before publication would surely lead back to an “Old Boys Club” of requesting preferential reviewers who may give a less stringent review of the manuscript. Publishing the reviewer’s details in association with the paper strikes me as a very good way of circumventing misconduct in the review process –any later error or retraction of the paper would have repercussions for the reviewer(s) as well as the Authors. This would obviously have much more impact if reviewers were listed on a league table or “reviewer index” that also formed part of the measure of Professional Competence (which is presently based largely on publication record) that funding agencies require.

Comments are closed.