Peer-to-peer

This post is by Myles Allen of the University of Oxford:
As a vocal supporter of the traditional system of scientists communicating through peer-reviewed channels ( Nat. Geosci. 1, 209; 2008 and associated debate at Peer-to-Peer), I was hesitant about writing a critical Commentary on the Feature 'A safe operating space for humanity' by Johan Rockström et al in the 24 September issue of Nature (Nature 461, 472-475; 2009) in a non-peer-reviewed forum. The Nature and Nature Reports Climate Change editors had clearly thought through this argument: the Feature was not itself peer-reviewed, so no golden rules would be broken in publishing a series of commentaries alongside it in Nature Reports Climate Change.
The problem is that packing the point into a few hundred words, and slipping into the usual bloggers' trap of feeling you have to shout loudly on the internet or no-one will listen, means that the end result reads very black-and-white. Corresponding with Johan Rockström over the past week, it emerges we agree on far more than the tone of my Commentary, 'Tangible targets are critical' (Nat. Rep. Climate Change doi:10.1038/climate.2009.95), probably implies. I understand that Rockström et al. had not originally intended to make the link between a six-degree climate sensitivity, the two degrees target and 350 p.p.m. a focal point (as I read it) of their Feature. Likewise, they observe, reasonably enough, that limiting cumulative carbon dioxide emissions to one trillion tonnes of carbon is just another way of framing the climate boundary, with (as I acknowledge) remarkably similar implications to 350 p.p.m.
If this had been done the old-fashioned way, Rockström et al. would almost certainly have had to qualify their reliance on a six-degree climate sensitivity in the course of the usual to-and-fro with referees. And I in turn would have toned down a lot of my objections. The end result would undoubtedly have been blander on both sides, but would that really have been much of a loss? Following what was, for me, an experiment, I still feel it is very much an open question whether scientific communication in general benefits from direct publication rather than allowing rough edges to be smoothed off through traditional peer-review.
Myles Allen is at the University of Oxford, Parks Road, Oxford, OX1 3PU, UK.
-----------------------------
Context:
Johan Rockström and his co-authors argue in a Nature Feature that to avoid catastrophic environmental change, humanity must stay within defined 'planetary boundaries' for a range of essential Earth-system processes. If one boundary is transgressed, then safe levels for other processes could also be under serious risk, they caution. Seven expert commentaries respond to this proposal in Nature Reports Climate Change, one of which is by Myles Allen. All these articles can be accessed from this index page.
There is an associated Nature podcast in which Johan Rockström is interviewed and editor Ehsan Masood provides further analysis.
Further reading:
A longer paper upon which the Nature Feature is based, 'Tipping towards the unknown', is available at the Stockholm Resilience Centre.
Nature news special: The road to Copenhagen.

Posted by Maxine Clarke on September 23, 2009
Categories: Quality and value | Permalink | Comments (3)

Manuscript peer reviewing is at the heart of the scientific system, but it seems that these duties are often not properly (if at all) recognized by universities, funding agencies or even the rest of the scientific community. This is the main message of the September Editorial in Nature Structural & Molecular Biology, 'The Unsung Reviewer' (16, 899; 2009) The Editorial notes:
"Scientists wear many hats these days. They prepare and teach classes and sit on various committees. Then there are the multiple activities directly related to their research, including grant writing, mentoring students and postdocs, attending conferences, writing papers and reviewing manuscripts. All these duties can take a considerable amount of time and effort and most are recognized as worthy contributions by funding agencies, universities and research institutions when evaluating a scientist's performance. On the other hand, peer reviewing papers seems to be the Rodney Dangerfield ("I get no respect!") of a scientist's duties."
A new report by the Science for Policy project of the US Bipartisan Policy Center notes that peer-review is essential for the scientific system. Journals such as NSMB (and the other Nature journals) would not agree from their own perspective with one of the report's conclusions, that "peer-review is no longer assumed to be a professional obligation". As the Editorial points out, our journals have very broad reviewer pools (running into many tens of thousands for the heavily oversubscribed weekly title Nature, for example), and the editors know and very much appreciate the work that these reviewers put into improving submitted manuscripts.
The SPP report also identifies a need to increase the number of scientists who participate in peer review for federal agencies, making suggestions such as listing such service on grant applications or even making it a requirement for funding. In addition, it urges journals to run a quality system, for example by providing the peer-reviewers with feedback (which the Nature journals do, and which is appreciated by our reviewers - for one reason, because a reviewer can see his or her report in the context of those of the other reviwers of the manuscript). The NSMB Editorial concludes:
"Peer review is often compared to jury duty, a chore that one has to do once in a while as a service to the community. But reviewing manuscripts can be enjoyable and useful at any career stage. True, it takes time, but as one reviewer aptly put it: "I still learn do's and don'ts from reviewing papers, as well as just getting useful information, so it is time spent pretty well." It is also true that one does not get much open recognition from it, and there are limitations on what the journals can do without compromising the reviewers' anonymity. We have in the past publicly acknowledged our reviewers at the end of the year, and will do so again this year. The SPP report stresses that it is important to stress that "peer reviewing manuscripts should be an expected and appreciated aspect of a scientist's career." Of course, we do have a vested interest in this, but we firmly believe that it's important to cultivate a vibrant 'reviewer culture'."

Nature journals' peer-review policies and guidelines.
More on peer review from the Nature journals.

Posted by Maxine Clarke on September 14, 2009
Categories: Quality and value | Permalink | Comments (0)

We received a question at the Nature Network 'Ask the Editor' forum which I thought readers of this blog might find useful.

Q: I once reviewed several manuscripts for a good journal, but it is on behalf of my Ph.D boss. Now, I am a postdoc and wonder how to become a reviewer? Thanks for any suggestions.

A. At Nature, we ask our peer-reviewers to identify anyone who helps them with their review. (The reviewers promise to keep the ms confidential, in advance of being sent it, and undertake to ensure that anyone they show it to also keeps it confidential). Therefore, we often discover (and regularly use) good new reviewers by this method. Many senior reviewers ask junior colleagues to review a ms as part of their mentoring, and are very good at assigning credit to these junior colleagues.
I suggest that if you help your boss or another colleague with his or her reviews in future, you ask that person to name you as a collaborator when he/she sends the journal the review, so your name gets known.
If you have been a co-author on your boss’s papers, you could contact the journal that published the work and offer to review (providing the information that you’ve published in that journal).
I think that as you publish in your own right, journal editors will get to know of you and start to ask you to review for them.
If you meet any journal editors at conferences you attend, you could let them know that you’d be interested in peer-reviewing for their journal.
Good luck!

Posted by Maxine Clarke on September 07, 2009
Categories: Journals, Peer Review Feedback | Permalink | Comments (0)

This is the text of a Correspondence published in Nature (460, 949; 2009) by Conor O'Carroll of the Irish Universities Association:

Your News story 'Italy outsources peer review to NIH' (Nature 459, 900; 2009) highlights a problem common to many countries with a small population of research scientists. Ireland can be held up as a successful model in addressing this problem because, over the past eight years, funding agencies have moved to fully international peer review.
A few years ago, important research and development ventures were set up with a new infrastructure to attract talented people from abroad. The use of only Irish peer reviewers to allocate millions of research euros to a small number of universities could not stand up to the principles of objectivity, transparency and perceived fairness and would have led to conflicts of interest. Despite initial opposition, exclusively international review is now accepted; researchers want to be benchmarked internationally as well as nationally.
The typical process for research evaluation in Ireland is to consult four or five reviewers by mail for each proposal. Proposals are then assessed by a panel of invited experts, who meet in Ireland. Reviewers may be sourced through international funding agencies, or by letting applicants nominate experts themselves.
Some Italian scientists in your News story express reservations. They may well have a point, as US reviewers will probably not have any detailed knowledge of how research is conducted in Italy. One approach is to have nationals involved, either as observers or in a formal non-voting role. For example, the Irish Health Research Board organizes international mail reviews and panels, but the chair of each is Irish. They cannot participate in selection, but ensure that the correct procedures are followed and can explain the national research-funding policy. International panel members appreciate this local input, which helps them think outside their own national funding system.
Reviewing criteria often include the quality of the project, the researchers and their institutions, and the social and economic impact of the research. It is important that international reviewers focus on the quality of the first two, as the standing of institutions and the probable impact of a project can be harder for them to evaluate. Also, they should not get involved in detailed budgetary considerations, as these are strictly national.
Things have changed radically in Ireland's research over the past ten years. In 2008, the country appeared for the first time in a list of 'Top countries in all fields' (ranked by citations per paper). We are now placed 19th, up from 36th place in 2003. I believe that international peer review played a significant part in this development.

Posted by Maxine Clarke on September 03, 2009
Categories: Policy and Public | Permalink | Comments (0)