Nautilus

Thomson Reuters have just announced the 2008 Impact Factors. Nature is the top journal in the multidisciplinary science category by all Thomson Reuters' new metrics: 5 year Impact Factor, Eigenfactor and article influence score. It is also the top of all journals in the Journal of Citation Reports (Thomson Reuters, 2009) listing (n=6,598) by Eigenfactor score. Here are the 2008 Impact Factors for the Nature journals that publish primary research:

Nature 31.434
Nature Biotechnology 22.297
Nature Cell Biology 17.774
Nature Chemical Biology 14.612
Nature Chemistry N/A
Nature Genetics 30.259
Nature Geoscience N/A
Nature Immunology 25.113
Nature Medicine 27.553
Nature Materials 23.132
Nature Methods 13.651
Nature Neuroscience 14.164
Nature Nanotechnology 20.571
Nature Photonics 24.982
Nature Physics 16.821
Nature Struct Molec Biol. 10.987

Posted by Maxine Clarke on June 24, 2009
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

Citations are an important component in the assessment of academic performance. Yet the growing literature, combined with format constraints of journals, encourage citation of reviews in preference to primary research. This diverts academic credit from the discoverer. (From the January Editorial of Nature Cell Biology 11, 1; 2009, free to access online)
This Editorial notes that of the research articles in the journal's previous issue, one-quarter of the citations were to reviews. Authors tend to cite reviews because of the print constraints of most journals, hence citing reviews often allows an author to use one citation to cite a group of primary research papers. Further, as Nature Cell Biology points out, "ISI (Thomson Scientific) continues to lump together citations of primary research papers and reviews. This has had a major impact on researchers and indeed journals: it boosts cumulative citations of the former, while providing papers that tend to be well-cited for the latter to beef up journal impact factors. We have argued previously for a disambiguation of primary and review citations.
An additional consideration is that in the current highly competitive world of cell biology, some researchers may be tempted to obfuscate the state of the field to enhance the apparent conceptual advance provided by their study. Rather than omitting a citation altogether, a less onerous approach may be to support a vague statement by citing a general review."
Nature Cell Biology is addressing these issues by increasing the reference limits in papers by 40%. Authors can now cite up to 70 references, rather than 50, in Articles; 40 instead of 30 in Letters, and 20 in Brief Communications. The journal strongly encourages authors to cite the primary literature where appropriate. Reviews are the only effective way to provide background information on whole fields or more focused topics with a considerable literature, but citations to the primary literature are essential for referring to specific findings.

See a related announcement in The EMBO Journal.

Posted by Maxine Clarke on January 19, 2009
Categories: Citation analysis | Permalink | Comments (0)

Nature Publishing Group (NPG) and Thomson Reuters are holding a joint seminar on publishing excellence and how to correctly interpret journal citation data on 23 January 2009 in Sydney, Australia. This seminar will go into detail about the use and misuse of impact factors along with a presentation by senior editor Leslie Sage on how to get published in Nature.
Four speakers will present on the following:
Antoine Bocquet, Associate Director, NPG Asia-Pacific:
Growth of Nature Publishing Group
Dr Leslie Sage, senior editor, physical sciences, Nature :
How to publish a paper in Nature
Dr Berenika M Webster, strategic business manager, Thomson Reuters Scientific, Asia Pacific:
About use and misuse of impact factor and other citation metrics
Dr Dugald McGlashan, associate publisher, Asian journals, NPG:
Developments in author and reader services in a changing publishing landscape
This seminar is free to attend and open to those interested in publishing in Nature titles and journal citation data.
See here for more information, details of the venue, and to reserve your place.

Posted by Maxine Clarke on January 08, 2009
Categories: Author benefits, Citation analysis, Writing | Permalink | Comments (0)

Publishing a paper in a journal has traditionally marked the end of a research project, but increasing numbers of academics are becoming interested in the publication process itself, according to the Editorial in the November issue of Nature Nanotechnology (3, 633; 2008). Many of these 'papers about papers' are concerned with citations and impact factors — researchers looking to get more citations for their papers are advised to write longer papers, work in teams or write the first paper on a topic (references in the Editorial). However, other authors have started to look behind the scenes at issues such as the changing nature of collaboration. The Editorial goes on to discuss some of these issues, including the h-index, a relatively recent yet controversial method of assessing a scientist's output.
Previous Nautilus posts about the h-index.
Previous Nautilus posts about authorship.
Previous Nautilus posts about citation analysis.

Posted by Maxine Clarke on November 18, 2008
Categories: Authorship, Citation analysis, Quality measures | Permalink | Comments (0)

James Testa, of Thomson Reuters, explains the organization's policy on the abuse of excessive self-citation in a Correspondence (Nature 455, 729; 2008):

In reply to Tomá Opatrný's Correspondence 'Playing the system to give low-impact journals more clout' (Nature 455, 167; 2008), we would like to point out that the practice of journal self-citation is not new. Thomson Reuters is aware that some journals have used extensive reference to their prior content to influence their citation metrics. The contribution of so-called journal self-citation has been included in Journal Citation Reports since it first appeared in 1975. In recent years, these data have been made more prominent to inform our subscribers of the effects of journal self-citation.
Thomson Reuters also reviews self-citation data for journals in which an exceptionally high self-citation rate artificially influences the impact factor and therefore belies its contribution to the scientific literature. The role of a journal's impact factor as an objective and integral measure becomes questionable at this level of self-citation.
Nine journals received no listing in Journal Citation Reports last year because of exceptionally high self-citation counts; their titles are listed in the Notices file on the journal's website. Journal self-citations will be reviewed each year. Once the problem of excessive self-citation resolves and we can publish an accurate impact factor, the titles will again appear in the journal. Each title continues to be indexed in other Thomson Reuters products.
The cause of the increased 2007 impact factor of Folia Phoniatrica et Logopaedica will be examined as part of the routine review of journal self-citations, and a decision will be made regarding continued listing of the journal in 2008's Journal Citation Reports.

Posted by Maxine Clarke on October 23, 2008
Categories: Citation analysis | Permalink | Comments (0)

A longer paper gathers more citations : Nature News
From Nature 455, 274-275 (2008):
Researchers could garner more citations simply by making their papers longer, a study seems to imply (K. Z. Stanek, Preprint at http://arxiv.org/abs/0809.0692; 2008).
In an analysis of 30,027 peer-reviewed papers published between 2000 and 2004 in top astronomy journals, astronomer Krzysztof Stanek of Ohio State University in Columbus found that the median number of citations increases with the length of the paper — from just 6 for papers of 2–3 pages to about 50 for 50-page papers.
There is, however, a limit to the benefits of size: citations start to tail off when papers reach lengths of 80 pages or so, perhaps because fewer people have the stamina to read them.
It is unexpected, says astronomer Jörg Dietrich of the European Southern Observatory headquarters in Germany, who recently conducted a similar analysis and found the same results but didn't publish them. "I expected that shorter papers would be cited more than longer ones," he says. "I assumed that people don't have the time to read long papers."
Papers of about 4 pages — the length of Letters in Astrophysical Journal and Astronomy and Astrophysics, which report brief summaries of work that is usually published in more detail later — fare better than papers 5–10 pages long. But brevity offers no such benefit for papers in the other two journals considered, Astronomical Journal and Monthly Notices of the Royal Astronomical Society, which do not have Letters.

Posted by Maxine Clarke on October 02, 2008
Categories: Citation analysis | Permalink | Comments (1)

Andreas Trabesinger, a senior editor at Nature Physics, writes in News and Views this month (Nature Physics 4, 677; 2008):

"The past is by no means definite. It is rather open", wrote the German historian of science Ernst Peter Fischer in Die Welt on 24 July 2006. In his column, Fischer introduced the "zeroth theorem of the history of science"; a discovery named after a person, the theorem says, did not originate from that person.
Take, for example, Avogadro's number, named after Amedeo Avogadro, who asserted that there is the same number of molecules contained in a given volume of any gas at the same temperature and pressure. However, it was not the Italian savant who first estimated that number, but the Austrian scientist Johann Josef Loschmidt. Indeed, German-language texts sometimes refer to the number 6.022 1023 as 'Loschmidt's number'. Much depends on who tells a story, and where and when. Fischer sees his zeroth theorem as an invitation to look with fresh eyes at the history of science, and in particular at how discoveries got their names.
That thought has now been picked up by J. David Jackson (Am. J. Phys. 76, 704–719; 2008). He has explored five examples from physics that illustrate Fischer's zeroth theorem, and discusses the broader issue of credit-giving, and where it gives rise to inappropriate attributions. Jackson's five examples take in various areas of physics, from the Dirac delta function to the Weizsäcker–Williams method of virtual quanta, to the Bargmann–Michel–Telegdi equation of spin dynamics. The journey includes encounters with big names such as Enrico Fermi or Nikola Tesla, but also with physicists whose biographies are far less commonly known, such as Oliver Heaviside, Llewellyn Thomas or Emil Wiechert. Their names are famous in some specific contexts, but little is known about their complete works."
Read more of the article at this link.

Posted by Maxine Clarke on September 11, 2008
Categories: Citation analysis | Permalink | Comments (0)

Nature Geoscience's September editorial (1, 563; 2008) broaches the subject of impact measures. From the editorial:

The ripples of the revolution in science evaluation have long reached the relatively uncompetitive backwaters of the geosciences. Indeed, Nature Geoscience received questions regarding its likely future impact factor before it was even accepted into Thomson Scientific's Web of Science in April this year. So here are a few thoughts on the topic from us, long before our own impact factor (due in 2010) may skew our perspective. Citation patterns vary hugely between disciplines. The impact factors of Nature and Science have ranged between 26 and 32 in the past few years. But a quick estimate, based on a sample of papers, suggests that geoscience papers in these journals score an impact factor of around 15 when evaluated on their own. This is high considering that the impact factors of journals publishing exclusively geoscience research have not exceeded 5 in the past several years. But far higher citation counts in the biological sciences drive up the statistics of journals that publish across disciplines. The timescales of the publication cycle in a field determine a journal's impact factor. These are defined as all citations in one year to citable content published in the two preceding years, thereby excluding all references more than two years from publication. This can be problematic for the slower-moving sciences. For example, the ten most cited papers in Geology in 2004 were collectively referred to about 1.5 times more often in 2007 than in 2006 — citations that have never entered the index. For geoscientists, taking guidance from impact factors alone would mean favouring interdisciplinary journals (whereas many biologists would, for the same reasons, favour their own disciplines). It would also lead to reading preferentially short-lived, quickly cited papers over those that develop more slowly — not necessarily a good idea. Other more time-consuming ways of assessing quality are therefore needed to supplement the quick and easy number check.

Posted by Maxine Clarke on September 04, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

The International Mathematical Union has released a report on the use of citations in assessing research quality. The report, Citation Statistics, is written from a mathematical perspective and strongly cautions against the over-reliance on citation statistics such as impact factor and h-index. The belief that these parameters are accurate, objective and simple, is unfounded.
It states that the objectivity of citations is illusory because the meaning of citations is not well-understood. Its meaning can be very far from ‘impact’. Although having a single number to judge quality is indeed simple, it can lead to a shallow understanding of something as complicated as research. Numbers are not inherently superior to sound judgments.
The report, written by mathematicians, promotes the sensible use of citation statistics in evaluating research and points out several common misuses of this widespread application of mathematics. The authors of the report recognize that assessment must be practical and that easily-derived citation statistics will be part of the process, but caution that citations provide only a limited and incomplete view of research quality. Research is too important, they say, for its value to be measured with only a single coarse tool.
(This is a precis of the press release accompanying publication of the report, see links above.)
Further discussion of the report, together with other matters related to citation and quality metrics, is taking place online at the Nature Network Citation in Science group, which all are welcome to join.

Posted by Maxine Clarke on July 02, 2008
Categories: Citation analysis | Permalink | Comments (0)

The 2007 Impact Factors are now out (published on 17 June 2008). The ten Nature Publishing Group journals with the highest Impact Factors are as follows:

1 NAT REV MOL CELL BIO 31.921
2 NAT REV CANCER 29.190
3 NATURE 28.751
4 NAT REV IMMUNOL 28.300
5 NAT MEDICINE 26.382
6 NAT IMMUNOLOGY 26.218
7 NAT GENETICS 25.556
8 NAT REV NEUROSCI 24.520
9 NAT REV DRUG DISCOV 23.308
10 NAT BIOTECHNOLOGY 22.848

The Impact Factors of the Nature journals that publish original research are:

1 NATURE 28.751
2 NAT MEDICINE 26.382
3 NAT IMMUNOLOGY 26.218
4 NAT GENETICS 25.556
5 NAT BIOTECHNOLOGY 22.848
6 NAT MATERIALS 19.782
7 NAT CELL BIOLOGY 17.623
8 NAT NEUROSCIENCE 15.664
9 NAT METHODS 15.478
10 NAT NANOTECHNOLOGY 14.917
11 NAT PHYSICS 14.677
12 NAT CHEM BIOLOGY 13.683
13 NAT STRUCT MOL BIOLOGY 11.085

(Nature Photonics and Nature Geoscience are not old enough to have been awarded an Impact Factor this year.)
Readers can create their own lists of journals by subject area, title, Impact Factor or publisher, at ISI Web of Knowledge.
There is a free-access account at the ThomsonISI website which explains how the Impact Factor for journals is calculated.
Discussion of the 2007 Impact Factors, and of citation in science in general, is taking place at the Nature Network Citation in Science group, which you are warmly invited to join.

Posted by Maxine Clarke on June 18, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Debbie Chaves of Wilfrid Laurier University, Waterloo, Ontario writes in Correspondence in the current issue of Nature (Nature 453, 719; 2008):

In his Correspondence 'Hall and Keynes join Arbor in the citation indexes' (Nature 452, 282; 2008 - see also Nautilus), Daniel Postellon describes the distinguished careers of Milton Keynes, Walton Hall and Ann Arbor. In the last case, I note that Professor Arbor has an h-index of 1 from the Web of Science database provided by Thomson Scientific's ISI Web of Knowledge. This is based on her five citations for the year 2007: two articles, two letters and one abstract.
An author search in the Web of Science reveals that Chevy Chase MD (not to be confused with Chevy Chase, Maryland) has co-authored a letter with Howard Kaplan (H. Kaplan Am. Sci. 96, 3; 2008). My own institution, Wilfrid Laurier University, is also an author (S. Cadell et al. J. Palliat. Care 23, 273–279; 2007).
Irrespective of how these errors are created, the rising use of systems in which citation information moves directly from the search of a database or citation index to a bibliographic management system, and then into a reference list, means that inexperienced students and researchers who are not savvy enough to detect these errors will propagate them further.
Vigilance is required by all users of citation indexes and databases.

Posted by Maxine Clarke on June 11, 2008
Categories: Citation analysis | Permalink | Comments (0)

The June editorial in Nature Neuroscience (11, 619; 2008) discusses the relationship between web traffic and citations. The journal’s preliminary analysis indicates that the number of downloads a paper receives immediately following its appearance online correlates very well with its citation frequency years after publication. Noah Gray, one of the Nature Neuroscience editors, has written a post at Action Potential, the journal’s blog, to provide more of the details behind the data and analysis, and to initiate discussion. He writes (edited for length):
Everyone has their own pet problem with impact factors, but despite these concerns, these numbers are typically used to rate the importance or prominence of a particular journal, and thus by proxy, the importance of the individual papers published within. This is a seriously flawed use of association, leading scientists to often equate the total number of citations with scientific impact, which can be fraught with problems. Searching for an alternative measure of impact that is perhaps free of the “bias of authority” (citing a paper because it is from a famous lab) or the “lemming bias” (citing a paper just because everyone else seems to do so whenever broaching a particular subject) led us to explore readership….
The “number of downloads” measure potentially provides a piece of an alternative solution for deciphering the impact of an individual paper. In this current scientific climate where tenure and grant funding decisions are influenced by flawed metrics like impact factor, it is important to make good use of all available technology in an attempt to realize a better system of measuring the scientific impact of any particular paper. This analysis is obviously preliminary and flawed in its own ways, but perhaps metrics such as paper downloads can find a place in a compilation of aggregated stats, painting a more accurate and informative picture of manuscript influence.
The Nature Neuroscience editorial.
The Action Potential post and discussion.
Nature Network Citation in science forum discussion.
Nature Network Citation in science group homepage.
Futher reading: Connotea bookmarks "citation"
Further reading: Connotea bookmarks "impact factor" : thanks to Bob O'Hara for this library.

Posted by Maxine Clarke on June 05, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Biji T. Kurien of the Oklahoma Medical Research Foundation writes (Nature 453; 450; 2008):
The Correspondence 'Give south Indian authors their true names' (Nature 452, 530; 2008) and earlier News Feature 'Identity crisis' (Nature 451, 766–767; 2008) are highly relevant to calculations of PubMed citations and h-index (the number n of a researcher's papers that have received at least n citations).
For example, I used to use the south Indian form of my name: T. Biji Kurien, with Biji being my personal name. I have seven publications cited incorrectly in PubMed as being by 'Kurien, T. B.', 'Bijikurien, T.' or 'Kurien, B.'. Four of these entries were cited often enough to be counted towards my h-index computation. As I had by then changed my name to conform with Western style, these publications unfortunately do not appear in the Web of Science or PubMed under my current name format. Consequently, my h-index ranking has fallen by 25% .
It is of paramount importance to adhere to a consistent name pattern right from the start, in order to maintain a correct list of publications in the public databases as well as the right h-index rankings.

Prabhu B. Patil, of the Centre for Cellular and Molecular Biology, Hyderabad, writes in the same issue of Nature (453, 450; 2008):
The Correspondence 'Give south Indian authors their true names' (Nature 452, 530; 2008), incorrectly states that people from the south do not traditionally have surnames.
I am from southern India and have a proper surname — as do all the families in my region. Besides Patil, surnames such as Naidu, Reddy, Rao and Gouda are common in the different states of southern India. One of the authors of the Correspondence has the surname Kutty.
Surnames have widely fallen into disuse because our fathers and forefathers avoided using them to prevent discrimination on grounds of caste.
It doesn't make sense in this case to use only an author's first name in scientific publications and to devise a special system to accommodate a different naming format. Instead, editors should encourage these authors to revive the use of their surnames.

Comments are welcome here and at Indigenus, the blog of Nature India.

Posted by Maxine Clarke on June 04, 2008
Categories: Authorship, Citation analysis | Permalink | Comments (0)

Daniel C. Postellon of the Helen DeVos Children's Hospital, Michigan, writes in Nature's Correspondence pages (Nature 452, 282; 2008):

The career of the non-existent author Ann Arbor is well-known to connoisseurs of computerized databases and citation indexes. Usually listed as the last author, she is sometimes credited with the academic degree "MI". Ann is not actually a person, but the city of Ann Arbor, Michigan, home of the University of Michigan. Her 'degree' is a misinterpretation of the abbreviation for Michigan: MI. She pre-dates online computerized databases, and was often listed in the paper edition of Index Medicus.
Ms Arbor now has a UK rival in the team of Walton Hall and Milton Keynes. Like her, they are usually listed as last authors. The online database Google Scholar lists them as co-authors of 46 publications, in addition to their solo work. Walton Hall is actually a building on the campus of the Open University in Milton Keynes. These 'authors' have a useful role to play: they can be used to check the accuracy of the databases and indexes.

Posted by Maxine Clarke on March 26, 2008
Categories: Citation analysis | Permalink | Comments (0)

Martin Fenner, on his Nature Network blog Gobbledygook, notes that The Deutsche Ärzteblatt , the official journal of the German Medical Association, will from this month be publishing an English version. The reason? So that the journal is more clearly indexed in databases such as PubMed, hence available to more readers, leading to more citations of journal articles, a better Impact Factor, and enhanced reputation of the journal. Martin's opinion is that although German was once an important scientific language, today only 2 per cent of articles indexed in Medline are in the language. "In the end", he writes, " it makes the exchange of ideas between scientists much easier if we can all use the same language. And Nature Network is a good example for this."
In the stimulating discussion arising from the post, Nicolau Werneck comments that "to this day there are a bunch of interesting words and expressions from German that came into the international scientific jargon in the last 2 centuries, such as gedankenexperiment, eigenvector and gestalt…We must fight. But not to forbid people from talking in english, or other imperialistic arrogant language, and certainly not to make them speak only in English. We must fight for the plurality of languages."
Nicholas Wigginton's view is that of someone considering a postdoc in a country where English is not an official language. "Although the science that the groups I am looking into publish everything in English, some operate their labs in the national language whereas others prefer their science to be done exclusively in science. I find this very interesting."

Posted by Maxine Clarke on January 25, 2008
Categories: Citation analysis, Format, Writing | Permalink | Comments (0)

Peter A. Todd of the National University of Singapore, and Richard J. Ladle of Oxford University, write in Correspondence this week (Nature 451, 244; 2008):
On 22 November, the Higher Education Funding Council for England announced that the assessment and funding of science-based disciplines will in future be "based on citation rates per paper, aggregated for each subject group at each institution".
Changes in performance indicators always strongly influence individual and institutional behaviour and 'citation game-playing' will no doubt become a staple of coffee-room conversation. What is less clear is how the citation practices of authors may influence bibliometric indicators.
Citation practices are known to be imperfect. The documented problems include excessive citation of an author's own work. Papers cited can be inappropriate or ambiguous in their support and, in some cases, the authors may not have read the papers they cite. Authors may form 'citation coalitions' within research networks. They may fail to provide citations to intellectual precursors or to work reporting conflicting conclusions. There are geographical and language biases. The increasing number of many-authored papers makes it impossible to have a clean-cut general metric in which one author is associated with one paper.
Taken together, these factors represent a problematic degree of error for the proposed bibliometric system of assessment. They place added responsibility on journal editors and reviewers as arbiters of appropriate author conduct.
Unfortunately, there are no simple solutions. Currently, identifying poor citation practices is not emphasized in the peer-review process, so perhaps journals could adopt a system of random citation audits, or periodically request evidence of citation appropriateness from authors. In reality, time constraints and the sheer volume of submissions to many journals mean that such measures are unlikely to be implemented soon.
Until referencing practices improve, we would argue that using citation rates to assess performance is fundamentally flawed.

Posted by Maxine Clarke on January 22, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

From this week's issue of Nature, a news story by Declan Butler "Free journal-ranking tool enters citation market" (Nature , 451, 6; 2008)
"A new Internet database lets users generate on-the-fly citation statistics of published research papers for free. The tool also calculates papers' impact factors using a new algorithm similar to PageRank, the algorithm Google uses to rank web pages. The open-access database is collaborating with Elsevier, the giant Amsterdam-based science publisher, and its underlying data come from Scopus, a subscription abstracts database created by Elsevier in 2004.
The SCImago Journal & Country Rank database was launched in December by SCImago, a data-mining and visualization group at the universities of Granada, Extremadura, Carlos III and Alcalá de Henares, all in Spain. It ranks journals and countries using such citation metrics as the popular, if controversial, Hirsch Index. It also includes a new metric: the SCImago Journal Rank (SJR)."
The article goes on to discuss the new metric, and to compare it with others, notably the dominant Impact Factor of Thomson Scientific. Thomson has set up a web forum to respond formally to an editorial in the Journal of Cell Biology, in which Mike Rossner and colleagues point to discrepancies between their "independent audit" calculations of Impact Factors based on Thomson data, and Thomson's own calculated Impact Factors from the same data.

Posted by Maxine Clarke on January 03, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Citation-based quality metrics were discussed on Nautilus earlier this year. One of those, the h (for highly cited) index, was covered recently in a News story, and is the subject of two Correspondence letters in the current issue of Nature.

Michael C. Wendel of Washington University Medical School writes (Nature 449, 403; 2007):
The h-index (the number n of a researcher's papers that have received at least n citations) may paint a more objective picture of productivity than some metrics, as your News story 'Achievement index climbs the ranks' (Nature 448, 737; 2007) points out. But for all such metrics, context is critical.
Many citations are used simply to flesh out a paper's introduction, having no real significance to the work. Citations are also sometimes made in a negative context, or to fraudulent or retracted publications. Other confounding factors include the practice of 'gratuitous authorship' and the so-called 'Matthew effect', whereby well-established researchers and projects are cited disproportionately more often than those that are less widely known. Finally, bibliometrics do not compensate for the well-known citation bias that favours review articles.

Clint D. Kelly and Michael D. Jennions of Australian National University, write (Nature 449, 403; 2007):
The h-index seems to be breaking away from the bibliometric pack, in the race to become a favoured measure of scientific performance ('Achievement index climbs the ranks' Nature 448, 737; 2007). However, if the h-index is to become an assessment tool commonly used by university administrators and government bureaucrats, those using it should be aware of its pitfalls.
As noted in your News story, tallying how many papers a researcher publishes (their productivity) gives undue merit to those who publish many inconsequential papers. But at least for ecologists and evolutionary biologists, the h-index is highly correlated with productivity.
This is worrisome, because the h-index is easily misconstrued as an equitable measure of research quality. We offer two examples.
First, female ecologists and evolutionary biologists publish fewer papers than their male counterparts, and they have significantly lower h-indices. Should administrators therefore conclude that men are better researchers? No. The gender difference vanishes if we control for productivity. It seems unlikely that this phenomenon is restricted to ecology and evolution.
Second, the h-index increases with age and using the ratio of the two can be problematic. Therefore, reliably comparing the performance of younger researchers with older ones is difficult.

Your views are welcome.

Posted by Maxine Clarke on October 04, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

Language Log: Citation Plagiarism?
From the Language Log entry linked above: "Plagiarism normally involves either the unacknowledged borrowing of someone else's idea or the unacknowledged borrowing of someone else's words. A third kind of plagiarism is, however, occasionally mentioned, namely the citation of a reference without acknowledging that it came from another source. If author Jones reads a paper by Smith and thereby learns of a paper by Doe and cites Doe without mentioning that he owes the reference to Smith, he has committed this kind of plagiarism, if plagiarism it be."
Bill Poser, author of the Language Log entry, goes on to argue why this type of plagarism is not, in his view, plagiarism, as there is no deception involved. The authors of the original reference may, in Dr Poser's view, "deserve more credit than they receive, but that is a different matter."

See here for the JISC plagiarism advisory service, which provides generic advice and guidance on all aspects of plagiarism prevention and detection to institutions, academics and students.

Posted by Maxine Clarke on August 22, 2007
Categories: Citation analysis, Ethics | Permalink | Comments (1)

What’s so wonderful about citations? asks Cambridge professor Peter Murray-Rust. Prof Murray-Rust has looked on Google Scholar for a paper which according to the publisher has more than 100,000 accesses, and found that it has 92 citations over the same period, which translates into one citation for every 1,000 (or so) downloads.
Prof Murray-Rust applied the same logic to himself. He was told by a publisher that his paper had been downloaded 6,000 times, so expected to find about 6 citations on Google Scholar -- but in the event found only one. "I’m not saying there are better ways - there probably aren’t", he writes. "If we make downloads a metric, then people will try to distort them. But let’s not take this [citation analysis] as seriously as we do."

Posted by Maxine Clarke on July 31, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

In a post entitled Copy-Editing and Citation-Linking , Michael Jubb of the Research Information Network compares the version of an article finalised by the author, and the version edited by the journal. An extract is provided here:

"Two recent articles in Learned Publishing, the journal published by the Association of Learned and Professional Society Publishers (ALPSP), have highlighted the issue. The first, by Wates and Campbell, looked at the changes made in copy-editing in articles published in a series of Blackwell journals. The second, by Goodman, Dowson and Yaremchuk, is in the current issue of Learned Publishing, but also, interestingly, through the University of Arizona’s repository. I have not tried to compare the two versions. It would be interesting to do so, not least because they found that as a result of publishers’ copy-editing “there were a number of differences between author-final and published versions that were ‘confusing’ and that sometimes the publisher version and sometimes the author version was the more confusing”.......
In an editorial ....Sally Morris also comments on the two articles, and lays considerable stress on the value that the publisher adds in the checking and formatting of references and the provision of citation linking via CrossRef..... the need to add DOI links is a relatively new one which I gather relatively few authors actually do themselves (and I was not guided so to do by the publishers of either of my recent articles)."
See here for the full article.
We would be interested to hear further feedback from authors about the editing and web services they received from Nature journals and NPG journals, to add to the regular "author experience" surveys we conduct.

Posted by Maxine Clarke on July 19, 2007
Categories: Author services, Citation analysis, Communication | Permalink | Comments (1)

Antonio G. Valdecasas and Uta Grothkopf write:
Leaving aside the adequacy of using citation counts to evaluate scientists' merit (as discussed for instance by Peter Lawrence in Nature 422, 259-261; 2003), there is the associated problem of exhaustive gathering of citations for certain kinds of publications, in particular monographs and book series that are not routinely covered by the Science Citation Index (SCI).
In countries like Spain where, as Lawrence points out, bureaucrats carry out scientists' assessment with "formulaic precision", there is, therefore, a risk of underrating this type of publication. Examples can be as extreme as to receive a dictum of almost 'no impact' for a monograph whose articles have been cited more than 150 times in journals included in the SCI, 19 times in non-SCI journals, 10 times in proceedings and which contains several articles in 'rare' languages such as Russian and Chinese. These citation results can be easily found through a simple search in Google Scholar. Not a bad citation number for a discipline (morphometrics) that has a mean number of citations for the period 1996-2005 of 8.5 (n= 1,408) citations per article, but most of them would be lost if one relies exclusively on the SCI.
Maybe the days of the SCI are numbered, as is already the case in disciplines such as astronomy, where alternative services are used. If impact is to be used as a metric that affects people directly, then databases like Google Scholar -- free, accessible to everybody, and non-discriminatory against languages other than English -- could provide a tool of universal coverage for bureaucrats and evaluation committees to discover the real impact of publications and hence to be less biased in the distribution of benefits.
Antonio G. Valdecasas
Museo Nacional de Ciencias Naturales, Madrid, Spain
Uta Grothkopf
European Southern Observatory, Garching, Germany.

Posted by Maxine Clarke on July 03, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (3)

The 2006 impact factors for the Nature journals can be viewed at the Author and Reviewers' website. Here is Nature Publishing Group's recent press release about the new (2006) Impact Factors.

In last week's Journal Citation Report (Thomson ISI), Nature Publishing Group (NPG) once again emerged as the high impact publisher. 12 of the top 30 titles are published by NPG. New titles have leapt to the top of their respective categories, and established titles retained their leading positions. Many of NPG's journals published on behalf of societies have also performed well.
NPG established a lead position in the physical science categories. Nature Physics, launched in October 2005, received its first impact factor of 12.04, securing its place as the leading monthly journal in physics. Nature Materials, now with its third impact factor, goes from strength to strength. Its impact factor leapt by 3.25 to 19.19, placing it first in three categories: materials, multidisciplinary; physics, applied; and chemistry, physical. Nature Chemical Biology also received its first impact factor of 12.41, ranking it in the top five monthly research journals in biochemistry and molecular biology.
Nature Methods has become the leading monthly journal in biochemical research methods, with its impact factor soaring from 6.74 to 14.96. NPG's open access journal, Molecular Systems Biology, received its first impact factor of 7.94. The seven journals from the Nature Reviews series, launched between 2001 and 2003, each rank as the leading monthly review journal in their specified category, significantly ahead of any competing review titles.
The Nature journals hold their leading positions across a range of categories. Nature Genetics and Nature Reviews Genetics hold the first and second positions in the category of genetics and heredity. Nature Medicine leads in medicine, research and experimental. Nature Reviews Immunology and Nature Immunology are the top monthly journals in immunology. Nature Reviews Neuroscience and Nature Neuroscience lead in the category of neuroscience. The category of cell biology is dominated by Nature journals - Nature Reviews Molecular Cell Biology (first), Nature Medicine (third), Nature Cell Biology (fifth) and Nature Structural & Molecular Biology (thirteenth).
NPG's academic and society journals also excelled, with most titles showing an increase in impact. The Journal of Investigative Dermatology secured an impact factor of 4.53 and is once again first among dermatology journals. Molecular Psychiatry ranks second in psychiatry with an impact factor of 11.8. Oncogene and Leukemia rank in the top 20 for oncology journals, and EMBO Journal and EMBO Reports also appear in the top 20 for cell biology journals. Cell Research's impact factor has risen to 3.4, making it the highest ranking journal in China and one of the highest in Asia. Cell Research is published on behalf of the Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences.
Nature retains its position as the most cited weekly science journal, with more than 390,000 citationa, an increase of almost 18,000 on last year's count. Nature continues to publish more articles than any other multidisciplinary journal.
"We are delighted with NPG's overall success this year, which sees the strengthening position of our new journals and the continued performance of our established titles. The authors that choose to publish with NPG are to be thanked for our strong results, says Annette Thomas, the company's Managing Director. NPG's service to societies remains strong, invigorating established titles with new content and new technology. We remain committed to high impact, innovative publishing."
This press release and all others can be viewed in the NPG press room.
The impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years. It is an independent measure calculated by Thomson/ISI (Institute for Scientific Information), Philadephia, USA.See the Author and Reviewers' website for more explanation of impact factors.

Posted by Maxine Clarke on July 02, 2007
Categories: Citation analysis, Publications | Permalink | Comments (1)

Scopus, the abstract and citation database of peer-reviewed literature and quality web sources, will incorporate the h-index soon. The h-index considers the publication records of an individual, the number of papers published over n years and the number of citations for each paper. The result is a single number, the h-index. To provide the user with additional clarity Scopus sys it will include visual aids that present a transparent overview of citation and publication patterns over time, revealing whether the h-index is dependent on a few highly cited papers or that the author’s papers have a relatively consistent volume of citations.
See here for a recent Nature Commentary on the h-index; and see here for a 2005 Nature news story "Index aims for fair ranking of scientists".

Posted by Maxine Clarke on May 01, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (2)

In the Editorial of the April issue of Nature Clinical Practice Rheumatology (3, 189; 2007), Editor in Chief Peter E. Lipsky writes: "The IF [Impact Factor] was envisioned over 50 years ago with the purpose of eliminating "the uncritical citation of fraudulent, incomplete or obsolete data by making it possible for the conscientious scholar to be aware of criticisms of earlier work" (Garfield E [1955] Science 122: 108–111). The IF has subsequently morphed into an institutionalized means of ranking the quality of scientific journals and, by implication, the individual articles published within them; for researchers, the IF influences employability, promotion, grant acceptability and bonus payments, and has been likened to a popularity contest."
Dr Lipsky discusses various flaws in the Impact Factor evaluation system, such as the lack of transparency of the formula by which the IF is calculated; that an erroneous but frequently cited article will bolster the ranking; disproportionate representation of review articles; and the differences in publication and citation frequencies in different disciplines. He asks whether his journal even actually wants an IF when it becomes eligible, concluding: "We do not feel that the current IF will reflect either our quality or our potential influence on clinical practice. Only if a new validated metric is developed that can evaluate the true quality and value of journal articles can we make real progress in improving the communication of new information in clinical medicine."
The full editorial is available at the journal's website or as a PDF here: Download file.
Comments are very welcome.

Posted by Maxine Clarke on April 27, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

From a Correspondence by Brian Haynes in Nature 446, 725; 2007.
ISI has recently delisted a number of publications from the Web of Science without informing the affected publishers or editors, or publishing a full list of the excisions. The motivation seems to have been to focus the Web of Science on journals and to move conference proceedings to another, little-known product, ISI Proceedings — notwithstanding the fact that many journals have special issues containing conference proceedings.
Proceedings of the Combustion Institute, an important archive in the multidisciplinary field of combustion dating back to 1928, is one of the affected publications. Because its peer-reviewed papers are presented at the biennial International Symposium on Combustion, they will no longer be listed in the Web of Science. According to ISI, the decision to exclude this publication "was not based on an evaluation of its importance to the community of scholars it serves".
This experience adds a new dimension to problems with excessive reliance on citation analyses. The Web of Science database itself is subject to unaccountable adjustments without scientific justification or regard to scientific importance.
Brian Haynes
The Combustion Institute, Pittsburgh, USA

Posted by Maxine Clarke on April 26, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

The United Kingdom Serials Group (UKSG), in association with the online usage metrics organisation COUNTER, is funding a study to explore how online journal usage statistics might form the basis of a new metric of journal quality, the "Usage Factor". The first stage of this project involved a series of interviews with various stakeholders, and the second, current stage involves a web-based survey designed to obtain the views of many more librarians and authors than was possible for the interview stage.

If you are an author of a publication in any of the Nature or Nature Publishing Group journals, we hope you can spare a few minutes to complete this important survey, which you can do by clicking on this link. The survey aims to:

* Discover what you think about the measures that are currently used to assess the value of scholarly journals (notably impact factors);
* Gauge the potential for usage-based measures;
* Provide an opportunity for you to suggest possible different, additional measures.

See here for the author survey. It will take less than 5 minutes to do; because it is due to close by 30 March (though it may be extended for a few days), please do visit the survey site now if you are interested in contributing your views and experience.

Posted by Maxine Clarke on March 28, 2007
Categories: Access, Citation analysis, Quality measures | Permalink | Comments (0)

Scopus, Elsevier’s abstract and citation database, has now added citation data, including abstracts, for many older papers, making it ever-closer to the rival Thomson’s Web of Science service. Nature Publishing Group titles are indexed by both databases, as well as by many other abstract&indexing services.

Since its launch two years ago, Scopus is now attracting publishers who want their older journals to be listed, including Springer, Nature Publishing Group, the American Physical Society, and the American Institute of Physics. Most of these publishers have digitized their content themselves back to the first issues of their journals. Older journals, such as Nature (first issue published on 4 November 1869, and freely available online), are still in the process of digitizing their very old content. Nature's digital archive currently goes back to 1950, with the issues back to 1869 to be added later this year. Nature New Biology and Nature Physical Sciences archives are also being digitized.

Scopus already contains 15 million pre-1996 records, and the new deal will add around seven million more. The new records will include full abstracts, traditionally a strength of Thomson’s Web of Science. Scopus does not necessarily intend to index older content in all areas, believing that 11 years of citation data is enough for some purposes - for example, to provide an overview of a publication for a grant or tenure application, so is prioritzing the digitization of older content accordingly.

In some cases, researchers may be interested in older content to see the citations that their own research has received over the years, or may use it to study historical trends. Whatever the needs of the reader, the addition of more search features and functions as the two organizations compete for customers can only be of benefit, especially taking into account features being introduced by the free-to-use service Google Scholar, also driving innovation and introduction of more reader services by the commercial providers.
See Nature Publishing Group abstracting and indexing page.

Posted by Maxine Clarke on January 26, 2007
Categories: Author services, Citation analysis, Publications | Permalink | Comments (0)

Citation analysis can loom large in a scientist's career. On pages 1003-1004 of the 21-28 December 2006 issue of Nature, Sune Lehmann, Andrew Jackson and Benny Lautrup compare commonly used measures of author quality. The mean number of citations per paper emerges as a better indicator than the more complex Hirsch index; a third method, the number of papers published per year, measures industry rather than ability. Careful citation analyses are useful, but Lehmann et al. caution that institutions often place too much faith in decisions reached by algorithm, use poor methodology or rely on inferior data sets.

Read the full text of the Commentary here (subscription or site licence required).

We welcome comments on this Commentary and on citation-based quality measures in general.

Posted by Maxine Clarke on January 04, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (6)