Of Schemes and Memes Blog

Journal metrics: handle with care

We have recently updated the journal metrics page for Nature Research to include an array of additional bibliometric data (www.nature.com/npg_/company_info/journal_metrics.html).  In addition to the traditional 2-year impact factor, we are now providing the 5-year impact factor, the immediacy index, the Eigenfactor score and the Article Influence Score. Whilst it is a measure that reflects a journal’s citations, the 2-year impact factor as an arithmetic mean of the citations per article can be disproportionately skewed by a minority of highly cited outliers.

For Nature, the Nature-branded research and reviews journals, and Scientific Reports, we have also provided the 2-year median, defined as the “the median number of citations received in 2015 for articles published in in 2013 and 2014.” Although the median citation is not subject to distortion by outliers like the 2-year impact factor, it should be noted that the median does not address another concern: that for multidisciplinary journals, such indicators measure across a broad swath of disciplines which have their own distinct citation rates. Nevertheless, we consider it to be a useful complement to the 2-year impact factor.

Given that this citation median is not a standard indicator, it is important to explain its derivation. To calculate the median, datasets for individual journals were downloaded from Web of Science in the third quarter of 2016.

  • For Nature and the Nature-branded research journals, articles published in 2013 and 2014 and categorized as “Article” or “Review” in Web of Science were selected.
  • For Nature Reviews journals, articles published in 2013 and 2014 and categorized as “Review” in Web of Science were selected. In addition, a small number of review articles that are categorized as “Article” in Web of Science were also included.
  • For Scientific Reports, articles published in 2013 and 2014 and categorized as “Article” in Web of Science were selected. In addition, a small number of research articles that are categorized as “Review” in Web of Science were also included.

For each dataset, (1) duplicated items were removed, (2) items that were not genuine Articles or Reviews were removed and  (3) items that were published outside the 2013-2014 timeframe as determined by issue date (for Nature and Nature-branded journals with issues) or date of online publication (for Nature Communications and Scientific Reports) were also removed. For each curated dataset (i.e. following steps 1-3 outlined above), the number of citations in 2015 to each item was taken from Web of Science and the median value calculated accordingly.

Thus, the median represents the midpoint number of citations that articles published in 2013 and 2014 received during 2015.

We believe that a broader array of citation-based metrics including the 2-year median citation will provide a more balanced perspective on journal performance and will be helpful to authors, readers and the community at large in assessing the quality of our journals.  At the article level, we have provided article-level metrics and Altmetric data since October 2012 for Nature journals. Our ambition for the future is to add alternative metrics at a journal level, to complement the citation-based metrics and the article level metrics.

We would emphasise that although this wider suite of metrics at the journal and article-level might provide interest and context, there can be no substitute for assessing each article on its own merits.

The Nature journals have a long record of publishing editorials on the limitations of the traditional journal impact factor.  The journal metrics page also provides links to a selection of these editorials from Nature journals and to the Nature special collection on Metrics (Nature 2010).

For better or worse, in survey upon survey, our authors cite impact factor as a primary consideration in driving decisions about where to publish their work, and whether to write for a journal.  This is despite  increasing calls for due caution in using the journal impact factor and a growing recognition that the over-reliance on the journal impact factor as an indicator for the quality of individual articles is damaging to the practice of science. There is no question that decreasing a reliance on the impact factor as an assessment metric is proving to be exceedingly challenging for journals, publishers, funders, research institutions and researchers.

Nevertheless, we hope to see increasing efforts from the community at large to develop mechanisms that  credit best practice in reproducible, robust research, putting the attention squarely where it belongs: on the researchers and their contributions to their fields.

Philip Campbell, Editor-in-Chief, Nature and Nature Research
Sowmya Swaminathan, Head of Editorial Policy, Nature Research


Comments are closed.