Peer-to-Peer

Journal of Biology adds a twist to peer-review

The Journal of Biology (8, 1; 2009) has announced an experimental policy of allowing authors of submitted manuscripts to opt-out of re-revew on occasions where the peer-reviewers require revisions, including the addition of data. In these cases, the journal will not publish the referees’ reports with the manuscript, but instead will publish an accompanying commentary.

This adds a new model of peer-review to those previously described in Nature‘s peer-review debate of 2006. Journals using unusual forms of peer-review system include Biology Direct, Atmospheric Chemistry and Physics, Signaling Gateway (a database publication), Electronic Transactions on Artificial Intelligence, and the BioMed Central (BMC) journalsBMC publishes Journal of Biology. Further discussion on systems of peer-review since 2006 is archived in a series of posts on this blog.

It will be interesting to see how the Journal of Biology experiment is received by readers and peer-reviewers (no doubt that it will be popular with authors). As well as the question of the accuracy of pubished papers whose authors have not addressed technical criticisms, there is also the question of motivation of peer-reviewers to write detailed reports, if they know their advice can, if the author wishes, go unheeded. Particularly strange to me is the decision by the journal not to publish the referees’ reports with the unrevised manuscript, but instead to publish an independent commentary. Will this commentary always accompany publication of incomplete manuscripts, or might it be delayed? Will it be written by one of the peer-reviewers, or if not, by someone who has access to the reports?

The Nature journals’ peer-review policy and procedures are described here. This web page contains an archive of free-to-access editorials in many Nature journals that discuss aspects of the peer-review system and process.

Comments

  1. Report this comment

    Matt Hodgkinson said:

    The peer review on the BMC-series journals isn’t that “unusual”. The biology journals have traditional anonymous review, while the medical journals operate under open peer review. Open peer review is unconventional but is essentially the same as standard peer review, only the authors know who reviewed their article (as do the readers, if the article is published).

  2. Report this comment

    Irene Hames said:

    As an editor for another journal, I’m restricting my comments to the new model of peer review. I’m commenting from the standpoint of a professional editor who has overseen the handling of around 14,000 manuscripts over the past 18 years and written a book on peer review. (I’m posting both here and at Nothing’s Shocking.)

    Noah and Maxine have made some very good points, and I agree with many of them. Noah’s post is a great summary of the issues and problems inherent in the new model, so I won’t go over what he’s already said. But this new model worries me greatly. When sweeping generalisations are made to promote a new model without providing evidence or references to back them up, that’s quite dangerous. Especially as outside of the main general science and highest impact journals, editors, and many editorial staff, come to their roles without any experience or training.

    One of my main concerns is that a crucial decision in the review process is being handed over to someone, the author, who is not impartial, but has a vested interest in a particular outcome. It’s the editor’s role to make that sort of decision, and to arbitrate between the various parties involved in the peer review process. If authors feel that reviewers are holding them to ransom, then the editor isn’t doing his or her job. It’s their role to assess what the reviewers say, taking into account many other pieces of information, and then advise the author on what needs to be done and what doesn’t, either because it’s totally unrealistic, or misguided. Peer review is a dialogue, and if an author doesn’t agree with what a reviewer says, or if a decision has been made on the basis of what the author perceives to be flawed information or reasoning, then they need to make a case and present it to the editor. A good editor is a reasonable editor. A dogmatic editor is a bad editor.

    It is very worrying that new data will be added without additional review if the authors decide they don’t want this, and Noah has explained why this is so. It’s quite alarming that the changes in such cases will be checked by the “editorial staff”. Will they be editors or assistants? Even if the former, there are many times when the judgement call can only be made by someone who is a specialist and, crucially, knows the current state of knowledge, problems and controversies in that field. In fast-moving areas it becomes even more important.

    It’s totally inappropriate for non-editor editorial assistants to check revisions. This happens. I have been contacted by concerned editorial assistants whose editors have asked them to do this and they’ve felt uncomfortable and out of their depth. I know from experience that the addition of a single sentence can have an enormous impact. Also, what about addition of things at inadequately monitored revision that will result in priority being inappropriately established? Or unwarranted claims of priority being made then?

    Part of the case made for the new model is to release more time for reviewers. But the new model is actually wasteful of reviewers’ time in one respect. If an author does opt to have their revised paper assessed by external reviewers again, it will go back to all the reviewers. But this shouldn’t automatically happen. It’s for the editor to step in and, based on the reviewers’ original comments, decide which of the reviewers needs to be consulted. The various reviewers may have been focussing on different aspects of the papers, they may have been asked for specific advice, some may therefore have been basically happy with the work except for minor issues. Others, may have picked up on more serious or complex points, and they’re the ones whose advice needs to be sought.

    So much hinges on the publication of work – grant funding, promotion, personal and professional prestige – that it’s crucial that all decisions to publish are made as fairly as possible and impartially. I can’t see how this is compatible with letting authors have such an important say at such a critical point of the peer review process.

  3. Report this comment

    Phil Hand said:

    As very much not an editor, I’m not really in a position to comment on the technicalities of peer review. However, I will just wonder out loud whether it really does what it says on the tin – aren’t there studies that show peer review often/sometimes doesn’t catch major mistakes? It seems to me that what’s important is not the form of the mechanism (single pass/iterative review), but rather the existence of some mechanism. The type of editorial control isn’t going to affect whether scientists do good research or not; the fact that editorial control of some type exists does provide a motivation to maintain high quality.

    But more important than that, I thought was this: "The function of a

    scientific journal is to disseminate the

    results of research."

    Is this not increasingly untrue these days? The function of the internet is to disseminate results. I actually wonder if the function of journals now might not be precisely to provide a filter on the vast sea of results out there. But whatever I think, what we’re seeing here is surely a journal fighting against the encroachment of a new medium on their turf.

Comments are closed.