Of Schemes and Memes Blog

Peer Review Week 2019: Improving peer review quality through transparent, reproducible research

This guest blog comes from Sowmya Swaminathan, Head of Editorial Policy and Research Integrity for Nature Research

By the time a research study reaches the peer review process, many crucial decisions that affect the rigor of the study design, methodology, data collection, analysis and reporting have already been made. Nevertheless, by developing and implementing editorial policies and by providing a publishing infrastructure that supports publication of transparent reproducible research, editors, journals and publishers can help improve the published paper, adding value and quality to the peer review and publication process.

Broadly speaking, four pillars – policy, publishing infrastructure, advocacy and awareness, and collective action – have driven editorial and publishing innovation and furthered our mission to work in partnership with the research community to advance quality and integrity.  In this blog post I provide an overview and examples of the many initiatives undertaken at Nature Research to support publication of reproducible research.

Policy

Transparency is at the heart of our policies designed to improve the reproducibility of published research. We ask authors to report information about their experimental design, as well as to clearly identify and make their datasets, code and materials available, also making it easier for reviewers to access the information they need to assess the study appropriately. We strongly support open research practices such as sharing the underlying building blocks of the research article – data, code and protocols – through repositories.

We have found that policies centred on transparency have had an impact.  For example, independent studies have found that the Nature Research Life Science Reporting Summary, an instrument to support transparent reporting in life science articles, which we introduced in 2013, has improved reporting of statistics and other aspects of experimental design and analysis [1,2].

We recognize that what works for reporting in life sciences is often not applicable to many of the other disciplines. While we advocate for a minimum threshold for transparency across core aspects of data, code, and materials, we have also worked with experts to tailor approaches that are designed to meet field-specific needs, for example in areas of photovoltaics and photonics research.

Data availability is another area where implementing a policy focused on transparency has had clear benefits. Since 2016, when we introduced a mandatory data availability statement on all research articles published in Nature-branded journals, we have seen a rise in data sharing through public repositories across our journals, especially in the life sciences, and increased appreciation of the value of data sharing to underscore the integrity and credibility of published work in many disciplines.

Publishing Infrastructure

Designing an innovative peer review and publishing infrastructure that supports all aspects of publishing reproducible research is central to our overall vision for an open and transparent ecosystem. A robust technology infrastructure is also essential to drive large-scale adoption of best practice approaches by authors, reviewers and editors. Over the years, we have introduced a number of publishing innovations that have furthered our commitment to reproducibility. These include avenues for publishing data and protocols such as Scientific Data and Protocol Exchange, and new article formats like Data Descriptors and Registered Reports that focus on data and methodological rigour respectively, rather than the specific results.

More recently, three Nature Research journals have tested executable platforms for peer review and publication of code. Although the policy and practice of peer reviewing code has been in place at these journals for many years, powering the process through an executable platform sets the stage for a more seamless and scalable experience for authors, reviewers and editors.

Advocacy and awareness

Advocacy and awareness-raising in the broader research and publishing community are other important areas of engagement for us in advancing our commitment to integrity in research. In the pages of Nature and the Nature-branded journals, we have often highlighted and debated the many different, complex issues, challenges and solutions on the path to transparent, reproducible research including discipline-specific needs and barriers to reproducible research (for example, see recent discussions about reproducibility in nano-medicine and data and code sharing in physics).

Collective action

Shifting entrenched patterns of how research is conducted and published requires stakeholders across the research and publishing community to work collectively in the push for better practice. Nature Research journals are proud to have participated in and supported many such efforts to accelerate data sharing, advance best practice toward open and transparent research and align on minimum reporting standards.

We believe that our editors and journals have an important role to play in tackling the many issues that affect the quality and integrity of published research. Indeed, we feel privileged to be able to engage with a global and multidisciplinary research community and are committed to furthering the cause of transparent, reliable research with all the tools at our disposal.

Join the discussions during Peer Review Week: #QualityinPeerReview, #PeerRevWk2019 #PeerReviewWeek

References:

  1. The NPQIP Collaborative group, Did a change in Nature journals’ editorial policy for life sciences research improve reporting? BMJ Open Science 2019;3:e000035. doi: 10.1136/bmjos-2017-000035
  2. Han S, et al. (2017) A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review. PLoS ONE 12(9): e0183591. https://doi.org/10.1371/journal.pone.0183591

Comments

Comments are closed.