Antonio G. Valdecasas and Uta Grothkopf write:

Leaving aside the adequacy of using citation counts to evaluate scientists’ merit (as discussed for instance by Peter Lawrence in Nature 422, 259-261; 2003), there is the associated problem of exhaustive gathering of citations for certain kinds of publications, in particular monographs and book series that are not routinely covered by the Science Citation Index (SCI).

In countries like Spain where, as Lawrence points out, bureaucrats carry out scientists’ assessment with “formulaic precision”, there is, therefore, a risk of underrating this type of publication. Examples can be as extreme as to receive a dictum of almost ‘no impact’ for a monograph whose articles have been cited more than 150 times in journals included in the SCI, 19 times in non-SCI journals, 10 times in proceedings and which contains several articles in ‘rare’ languages such as Russian and Chinese. These citation results can be easily found through a simple search in Google Scholar. Not a bad citation number for a discipline (morphometrics) that has a mean number of citations for the period 1996-2005 of 8.5 (n= 1,408) citations per article, but most of them would be lost if one relies exclusively on the SCI.

Maybe the days of the SCI are numbered, as is already the case in disciplines such as astronomy, where alternative services are used. If impact is to be used as a metric that affects people directly, then databases like Google Scholar — free, accessible to everybody, and non-discriminatory against languages other than English — could provide a tool of universal coverage for bureaucrats and evaluation committees to discover the real impact of publications and hence to be less biased in the distribution of benefits.

Antonio G. Valdecasas

Museo Nacional de Ciencias Naturales, Madrid, Spain

Uta Grothkopf

European Southern Observatory, Garching, Germany.