On Tuesday 24th March we introduced a small-scale, one month experiment on fast track peer review (up to 40 manuscripts maximum), which would enable authors to receive a first decision within three weeks of passing our quality control checks. You can read our original post on this here.
The trial has been running for two weeks and we have a couple of weeks left to run on it. We received some feedback from Editorial Board Members, authors and the general scientific community, both positive and negative. Last week a group of Editorial Board Members wrote to us, and our response of 31st March is below. The Editorial Board Members have since raised some additional questions which we will answer shortly, as an update to this blog post.
Rubriq are also updating their own website with more information about the trial and their procedures. We continue discussions with our Editorial Board Members and other members of the scientific community, and plan to feed back on the results of the trial once it has concluded and we have had time for analysis.
E-MAIL DATED 31st MARCH 2015
Thank you for your thoughtful letter regarding our experiment on peer review with Rubriq. I appreciate your getting in touch, and I’m happy to clarify some aspects of the trial.
This is a very small scale pilot study involving a few manuscripts (approx. 40) over the course of a few weeks. We have assessed the quality of service that Rubriq provides and feel confident that the peer review reports they will deliver will be of a comparable standard to our own. This was a crucial factor in our decision to work with them, as we place enormous importance on the quality of our peer review process.
Reviewer selection is carried out by a team of Peer Review Coordinators at Rubriq following criteria agreed by Scientific Reports in-house Editors so that they match our own – including those used to exclude potential conflicts of interest. The reviews that are solicited by Rubriq first go through internal vetting by the Peer Review Coordinators to ensure that the reviewers have provided detailed, actionable evaluations. These reviews are then transferred to Scientific Reports (with the identities and affiliations of the reviewers) where they are then evaluated by the Scientific Reports in-house Editors.
Appeals will handled by Scientific Reports following standard procedures.
NPG is first and foremost here to serve scientists. As scientists our method is to run experiments, measure the results, learn and adapt. Testing and evolving the peer review process is something we’ve embraced over many years at NPG. The decision to conduct this pilot study was taken after careful consideration – in a 2014 survey of over 30,000 NPG researchers, authors told us that they want us to innovate when it comes to peer review:
· 70% authors are frustrated with the time peer-review takes
· 77% think traditional peer review could be made more efficient
· 67% think publishers should experiment with alternative peer-review methods
Reviewers involved in the fast-track trial will be paid by Rubriq on condition they return their reports within an agreed timeframe – there should not be any difference in quality between fast track and standard reviewer reports.
We know with any innovation that there will be as many challenges as opportunities and so we will be closely monitoring the trial to check for any differences in metrics, other than faster time to first decision, between fast-track and standard submissions to ensure that we are not introducing any biases.
This small scale study will not affect our standard peer review service so authors who are unable to afford the fast-track option will not be impacted and our usual waiver policy will continue to be applied to non-fast track submissions.
We’re committed to exploring, learning, and better understanding the needs and choices of our authors. We will be carefully considering feedback from across the whole research community, including our Editorial Board Members, when deciding whether to extend the pilot study.
As members of the Scientific Reports editorial board, I value your feedback and I would be happy to discuss this further if helpful.
With all best wishes,
UPDATE 8th April 2015
We have responded to the following questions raised by 19 editorial board members of Scientific Reports. These responses were sent today as part of an email correspondence, but they may help to answer questions from others. As promised, we are sharing them here.
We have recently conducted a successful private parallel run of peer-review outputs comparing Rubriq with Scientific Reports. We have worked closely with Rubriq to be confident that the reports they provide are as rigorous as we would expect from our own Scientific Reports reviewers.
I’d like to take this opportunity to address the specific questions on your website:
Who are the peer review coordinators at Rubriq?
The peer review co-ordinators are PhDs who work as in-house members of Rubriq staff. Based on the topic and the techniques used in the research, the peer review co-ordinators match manuscripts to reviewers with a published track record of expertise in these areas. They do not make decisions on the manuscript but perform a review of each reviewer’s report to ensure that the review is complete and thoughtfully executed.
What are their scientific and academic credentials?
The minimum requirement for a peer review co-ordinator is a PhD in the biomedical sciences.
The external peer reviewers they invite are all active, publishing researchers with a minimum of a doctorate-level degree in the relevant field.
Do they have the necessary expertise to assign reviewers?
Each peer review co-ordinator has at least three years’ experience in identifying and recruiting appropriate peer reviewers for manuscripts.
Why is there only internal vetting of peer reviews?
The standard peer review process at Scientific Reports involves editorial board members choosing peer reviewers, and then making editorial decisions based on reports provided by these reviewers. For the purposes of this small-scale pilot (which was not intended to be scalable in its current format) peer reviewers are chosen by Rubriq. The editorial decisions, based on peer reviewers’ reports, are made by PhD-qualified, in-house Scientific Reports editors. This ensures we can deliver a first decision within three weeks.
The trial has been running for two weeks and we have a couple of weeks left to run on it. We have received both positive and negative feedback, and plan to summarise key findings of the trial once it has concluded and we have had time for analysis.