Peer-to-Peer

Mentors of tomorrow

This is the full text of an Editorial in today’s issue of Nature (447, 754; 14 June 2007), on which we welcome your responses in the comments section to this post.

Everyone knows bad peer review when they come across it — but too few are nurturing good referees.

There is nothing more infuriating: you are an experienced scientist who has sent one of your best-ever papers to a journal, and what do you get back? A set of referees’ comments that appals you. One reviewer asserts that the work is simply uninteresting and insufficiently original. Another displays wilful bias in relating their criticisms to results by a competitor whose outlook differs radically from yours. And a third has unreasonable expectations of what should be achieved. Not only are you upset, but your student co-author is devastated.


Such a trio of inadequate referees’ reports would rightly make an author doubt the credibility of the editor, who has a duty to ensure fair play. But they might also give the offended author reason to reflect a little. What right has he or she to expect a high quality of peer review? What training is being given in his or her own lab to ensure that the next generation understands how to do a good job of critically appraising others’ work? And as the pressures on researchers grow — bureaucracy from institutions and funding agencies, incentives to apply the outcomes of research — the very motivation to do a conscientious job of peer review is itself under pressure.

The fact is that the skills required to be a conscientious peer reviewer cannot and should not be taken for granted in young scientists. The culture of peer review is hugely variable — some heads of research groups take such training seriously, others do nothing. A more robust culture of good refereeing needs to be sustained — and that needs strong mentors.

Nature has for some years campaigned for good mentoring in the laboratory, particularly through a set of awards introduced in 2005 (the entry period for this year’s awards, which are for researchers in South Africa, closes this week). On page 791, we publish a guide to all aspects of mentoring, derived from the diverse and very specific endorsements of good practice received through the nominations process.

Many of the goals of mentoring are self-evident. But the explicit fostering of ethics in the lab is rare. And ethical practice includes doing justice to other researchers in critically assessing their work.

So what are the elements of good peer review? The most important aspect is attitude, which should be one of constructive objectivity. This includes not only avoiding scientific bias, but also leaving behind any preconceptions about the labs in which the work was done.

The content should be constructive, too. To summarize the paper and highlight its strengths and essential significance not only shows a proper sense of engagement, but also helps both author and editor to benefit from a fresh perspective. But it is in making critical comment that a referee genuinely adds value, provided that such comments are constructively and collegially expressed. If there are experimental weaknesses or alternative interpretations, it helps to suggest experiments to strengthen the case or resolve ambiguities. If inadequate credit is given to previous work or if the paper fundamentally lacks originality, it is a good idea to provide appropriate references.

This may all seem obvious to the experienced reviewer. (Readers can find more on good peer review, and can comment on it and on this Editorial, at http://blogs.nature.com/peer-to-peer.) But it is only by careful oversight of a young scientist’s attempts at reviewing real papers that the benefits of this experience can be passed on. Nature and Nature journals encourage the involvement of younger scientists by a referee, provided the authors’ need for confidentiality is respected and the additional reviewer identified. Such mentoring should be routine in research team leadership, if standards are to be kept high.

See here for Nature‘s guide for mentors.

Comments

  1. Report this comment

    Pedro Beltrao said:

    Maybe this could be another reason to have the referee comments openly available. I think learning from critically reading reports could also help us in finding our own guidelines to what a good referee report is, what is expected, etc.

    In bioinformatics, much of the work is quick enough to reproduce (once the results/methods are provided) that a referee could even test a control, instead of suggesting it to the authors. Should we do it if we can? It is hard to know.

  2. Report this comment

    Lingbo Li said:

    This is a great article as to how constructive peer reviews may greatly help the advance of scientific research and nurture the next generation of scientists. However, tackling the underlying issues (such as lack of training, time, and incentive to make a good review) will take some time. In the short run, I think such issues may be partially improved if journals can provide some guidelines to help both the reviewers and the authors to focus on the key components of a constructive review.

    Standardized evaluation forms are used in many fields to assist efficient and productive evaluation. Therefore I think such formats may also be helpful for academic peer reviews, which are essentially a form of evaluation. For example, a standard peer-review form may consist the following categories: “Whether you think the study is suitable for this journal”, “the strength of the study”, “the weakness of the study”, “Proposed experiments to support the conclusion”, etc. These categories can be further divided into smaller areas, with more specific, but objective questions. For each paper to be reviewed, specific spaces can be saved for each of the figure/table for reviewers to fill in (or skip) specific comments. A well-designed questionnaire-like review form may save the reviewers time and energy to come up with (sometimes) a long letter by themselves. It may also help them to organize their thoughts in a more logic and objective way. It should not add any extra burden to the journal once the format is fixed. The clear questions/titles in each category will also help both the editor and authors to focus on the key messages of the review.

  3. Report this comment

    Anthony Robbins & Phyllis Freeman said:

    We concur that peer review would improve with mentoring of reviewers. Our experience with our public health policy quarterly led to the concept of AuthorAID, develomental editing mentoring for authors from developing world institutions. The response has been more than encouraging, with scientists from around the world offering to serve as mentors on promising writing projects. See the 5 Sept 2005 opinion piece on SciDev.Net and the editorials we have published in the Journal of Public Health Policy. http://www.palgrave-journals.com/jphp/

  4. Report this comment

    Anthony Robbins & Phyllis Freeman said:

    We concur that peer review would improve with mentoring of reviewers. Our experience with our public health policy quarterly led to the concept of AuthorAID, develomental editing mentoring for authors from developing world institutions. The response has been more than encouraging, with scientists from around the world offering to serve as mentors on promising writing projects. See the 5 Sept 2005 opinion piece on SciDev.Net and the editorials we have published in the Journal of Public Health Policy. http://www.palgrave-journals.com/jphp/

  5. Report this comment

    Anthony Robbins & Phyllis Freeman said:

    As co-editors of a small public health quarterly our experience has taught us the value of mentorship. In fact, almost four years ago we proposed the concept of AuthorAID, developmental editing help for researchers in developing world institutions who wish to publish their promising writing projects. To our pleasure, scientists around the world have offered to be mentors and encouraged us to make AuthorAID operational. See our 5 September 2005 opinion piece on SciDev.Net and additional items about AuthorAID that have appeared in the Journal of Public Health Policy: http://www.palgrave-journals.com/jphp/

    Anthony Robbins & Phyllis Freeman, co-editors, Journal of Public Health Policy

  6. Report this comment

    Anthony Robbins & Phyllis Freeman said:

    As co-editors of a small public health quarterly our experience has taught us the value of mentorship. In fact, almost four years ago we proposed the concept of AuthorAID, developmental editing help for researchers in developing world institutions who wish to publish their promising writing projects. To our pleasure, scientists around the world have offered to be mentors and encouraged us to make AuthorAID operational. See our 5 September 2005 opinion piece on SciDev.Net and additional items about AuthorAID that have appeared in the Journal of Public Health Policy: http://www.palgrave-journals.com/jphp/

    Anthony Robbins & Phyllis Freeman, co-editors, Journal of Public Health Policy

Comments are closed.