Nature Geoscience’s September editorial (1, 563; 2008) broaches the subject of impact measures. From the editorial:
The ripples of the revolution in science evaluation have long reached the relatively uncompetitive backwaters of the geosciences. Indeed, Nature Geoscience received questions regarding its likely future impact factor before it was even accepted into Thomson Scientific’s Web of Science in April this year. So here are a few thoughts on the topic from us, long before our own impact factor (due in 2010) may skew our perspective.
Citation patterns vary hugely between disciplines. The impact factors of Nature and Science have ranged between 26 and 32 in the past few years. But a quick estimate, based on a sample of papers, suggests that geoscience papers in these journals score an impact factor of around 15 when evaluated on their own. This is high considering that the impact factors of journals publishing exclusively geoscience research have not exceeded 5 in the past several years. But far higher citation counts in the biological sciences drive up the statistics of journals that publish across disciplines.
The timescales of the publication cycle in a field determine a journal’s impact factor. These are defined as all citations in one year to citable content published in the two preceding years, thereby excluding all references more than two years from publication. This can be problematic for the slower-moving sciences. For example, the ten most cited papers in Geology in 2004 were collectively referred to about 1.5 times more often in 2007 than in 2006 — citations that have never entered the index.
For geoscientists, taking guidance from impact factors alone would mean favouring interdisciplinary journals (whereas many biologists would, for the same reasons, favour their own disciplines). It would also lead to reading preferentially short-lived, quickly cited papers over those that develop more slowly — not necessarily a good idea. Other more time-consuming ways of assessing quality are therefore needed to supplement the quick and easy number check.