Nautilus

The International Mathematical Union has released a report on the use of citations in assessing research quality. The report, Citation Statistics, is written from a mathematical perspective and strongly cautions against the over-reliance on citation statistics such as impact factor and h-index. The belief that these parameters are accurate, objective and simple, is unfounded.
It states that the objectivity of citations is illusory because the meaning of citations is not well-understood. Its meaning can be very far from ‘impact’. Although having a single number to judge quality is indeed simple, it can lead to a shallow understanding of something as complicated as research. Numbers are not inherently superior to sound judgments.
The report, written by mathematicians, promotes the sensible use of citation statistics in evaluating research and points out several common misuses of this widespread application of mathematics. The authors of the report recognize that assessment must be practical and that easily-derived citation statistics will be part of the process, but caution that citations provide only a limited and incomplete view of research quality. Research is too important, they say, for its value to be measured with only a single coarse tool.
(This is a precis of the press release accompanying publication of the report, see links above.)
Further discussion of the report, together with other matters related to citation and quality metrics, is taking place online at the Nature Network Citation in Science group, which all are welcome to join.

Posted by Maxine Clarke on July 02, 2008
Categories: Citation analysis | Permalink | Comments (0)

The 2007 Impact Factors are now out (published on 17 June 2008). The ten Nature Publishing Group journals with the highest Impact Factors are as follows:

1 NAT REV MOL CELL BIO 31.921
2 NAT REV CANCER 29.190
3 NATURE 28.751
4 NAT REV IMMUNOL 28.300
5 NAT MEDICINE 26.382
6 NAT IMMUNOLOGY 26.218
7 NAT GENETICS 25.556
8 NAT REV NEUROSCI 24.520
9 NAT REV DRUG DISCOV 23.308
10 NAT BIOTECHNOLOGY 22.848

The Impact Factors of the Nature journals that publish original research are:

1 NATURE 28.751
2 NAT MEDICINE 26.382
3 NAT IMMUNOLOGY 26.218
4 NAT GENETICS 25.556
5 NAT BIOTECHNOLOGY 22.848
6 NAT MATERIALS 19.782
7 NAT CELL BIOLOGY 17.623
8 NAT NEUROSCIENCE 15.664
9 NAT METHODS 15.478
10 NAT NANOTECHNOLOGY 14.917
11 NAT PHYSICS 14.677
12 NAT CHEM BIOLOGY 13.683
13 NAT STRUCT MOL BIOLOGY 11.085

(Nature Photonics and Nature Geoscience are not old enough to have been awarded an Impact Factor this year.)
Readers can create their own lists of journals by subject area, title, Impact Factor or publisher, at ISI Web of Knowledge.
There is a free-access account at the ThomsonISI website which explains how the Impact Factor for journals is calculated.
Discussion of the 2007 Impact Factors, and of citation in science in general, is taking place at the Nature Network Citation in Science group, which you are warmly invited to join.

Posted by Maxine Clarke on June 18, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Debbie Chaves of Wilfrid Laurier University, Waterloo, Ontario writes in Correspondence in the current issue of Nature (Nature 453, 719; 2008):

In his Correspondence 'Hall and Keynes join Arbor in the citation indexes' (Nature 452, 282; 2008 - see also Nautilus), Daniel Postellon describes the distinguished careers of Milton Keynes, Walton Hall and Ann Arbor. In the last case, I note that Professor Arbor has an h-index of 1 from the Web of Science database provided by Thomson Scientific's ISI Web of Knowledge. This is based on her five citations for the year 2007: two articles, two letters and one abstract.
An author search in the Web of Science reveals that Chevy Chase MD (not to be confused with Chevy Chase, Maryland) has co-authored a letter with Howard Kaplan (H. Kaplan Am. Sci. 96, 3; 2008). My own institution, Wilfrid Laurier University, is also an author (S. Cadell et al. J. Palliat. Care 23, 273–279; 2007).
Irrespective of how these errors are created, the rising use of systems in which citation information moves directly from the search of a database or citation index to a bibliographic management system, and then into a reference list, means that inexperienced students and researchers who are not savvy enough to detect these errors will propagate them further.
Vigilance is required by all users of citation indexes and databases.

Posted by Maxine Clarke on June 11, 2008
Categories: Citation analysis | Permalink | Comments (0)

The June editorial in Nature Neuroscience (11, 619; 2008) discusses the relationship between web traffic and citations. The journal’s preliminary analysis indicates that the number of downloads a paper receives immediately following its appearance online correlates very well with its citation frequency years after publication. Noah Gray, one of the Nature Neuroscience editors, has written a post at Action Potential, the journal’s blog, to provide more of the details behind the data and analysis, and to initiate discussion. He writes (edited for length):
Everyone has their own pet problem with impact factors, but despite these concerns, these numbers are typically used to rate the importance or prominence of a particular journal, and thus by proxy, the importance of the individual papers published within. This is a seriously flawed use of association, leading scientists to often equate the total number of citations with scientific impact, which can be fraught with problems. Searching for an alternative measure of impact that is perhaps free of the “bias of authority” (citing a paper because it is from a famous lab) or the “lemming bias” (citing a paper just because everyone else seems to do so whenever broaching a particular subject) led us to explore readership….
The “number of downloads” measure potentially provides a piece of an alternative solution for deciphering the impact of an individual paper. In this current scientific climate where tenure and grant funding decisions are influenced by flawed metrics like impact factor, it is important to make good use of all available technology in an attempt to realize a better system of measuring the scientific impact of any particular paper. This analysis is obviously preliminary and flawed in its own ways, but perhaps metrics such as paper downloads can find a place in a compilation of aggregated stats, painting a more accurate and informative picture of manuscript influence.
The Nature Neuroscience editorial.
The Action Potential post and discussion.
Nature Network Citation in science forum discussion.
Nature Network Citation in science group homepage.
Futher reading: Connotea bookmarks "citation"
Further reading: Connotea bookmarks "impact factor" : thanks to Bob O'Hara for this library.

Posted by Maxine Clarke on June 05, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Biji T. Kurien of the Oklahoma Medical Research Foundation writes (Nature 453; 450; 2008):
The Correspondence 'Give south Indian authors their true names' (Nature 452, 530; 2008) and earlier News Feature 'Identity crisis' (Nature 451, 766–767; 2008) are highly relevant to calculations of PubMed citations and h-index (the number n of a researcher's papers that have received at least n citations).
For example, I used to use the south Indian form of my name: T. Biji Kurien, with Biji being my personal name. I have seven publications cited incorrectly in PubMed as being by 'Kurien, T. B.', 'Bijikurien, T.' or 'Kurien, B.'. Four of these entries were cited often enough to be counted towards my h-index computation. As I had by then changed my name to conform with Western style, these publications unfortunately do not appear in the Web of Science or PubMed under my current name format. Consequently, my h-index ranking has fallen by 25% .
It is of paramount importance to adhere to a consistent name pattern right from the start, in order to maintain a correct list of publications in the public databases as well as the right h-index rankings.

Prabhu B. Patil, of the Centre for Cellular and Molecular Biology, Hyderabad, writes in the same issue of Nature (453, 450; 2008):
The Correspondence 'Give south Indian authors their true names' (Nature 452, 530; 2008), incorrectly states that people from the south do not traditionally have surnames.
I am from southern India and have a proper surname — as do all the families in my region. Besides Patil, surnames such as Naidu, Reddy, Rao and Gouda are common in the different states of southern India. One of the authors of the Correspondence has the surname Kutty.
Surnames have widely fallen into disuse because our fathers and forefathers avoided using them to prevent discrimination on grounds of caste.
It doesn't make sense in this case to use only an author's first name in scientific publications and to devise a special system to accommodate a different naming format. Instead, editors should encourage these authors to revive the use of their surnames.

Comments are welcome here and at Indigenus, the blog of Nature India.

Posted by Maxine Clarke on June 04, 2008
Categories: Authorship, Citation analysis | Permalink | Comments (0)

Daniel C. Postellon of the Helen DeVos Children's Hospital, Michigan, writes in Nature's Correspondence pages (Nature 452, 282; 2008):

The career of the non-existent author Ann Arbor is well-known to connoisseurs of computerized databases and citation indexes. Usually listed as the last author, she is sometimes credited with the academic degree "MI". Ann is not actually a person, but the city of Ann Arbor, Michigan, home of the University of Michigan. Her 'degree' is a misinterpretation of the abbreviation for Michigan: MI. She pre-dates online computerized databases, and was often listed in the paper edition of Index Medicus.
Ms Arbor now has a UK rival in the team of Walton Hall and Milton Keynes. Like her, they are usually listed as last authors. The online database Google Scholar lists them as co-authors of 46 publications, in addition to their solo work. Walton Hall is actually a building on the campus of the Open University in Milton Keynes. These 'authors' have a useful role to play: they can be used to check the accuracy of the databases and indexes.

Posted by Maxine Clarke on March 26, 2008
Categories: Citation analysis | Permalink | Comments (0)

Martin Fenner, on his Nature Network blog Gobbledygook, notes that The Deutsche Ärzteblatt , the official journal of the German Medical Association, will from this month be publishing an English version. The reason? So that the journal is more clearly indexed in databases such as PubMed, hence available to more readers, leading to more citations of journal articles, a better Impact Factor, and enhanced reputation of the journal. Martin's opinion is that although German was once an important scientific language, today only 2 per cent of articles indexed in Medline are in the language. "In the end", he writes, " it makes the exchange of ideas between scientists much easier if we can all use the same language. And Nature Network is a good example for this."
In the stimulating discussion arising from the post, Nicolau Werneck comments that "to this day there are a bunch of interesting words and expressions from German that came into the international scientific jargon in the last 2 centuries, such as gedankenexperiment, eigenvector and gestalt…We must fight. But not to forbid people from talking in english, or other imperialistic arrogant language, and certainly not to make them speak only in English. We must fight for the plurality of languages."
Nicholas Wigginton's view is that of someone considering a postdoc in a country where English is not an official language. "Although the science that the groups I am looking into publish everything in English, some operate their labs in the national language whereas others prefer their science to be done exclusively in science. I find this very interesting."

Posted by Maxine Clarke on January 25, 2008
Categories: Citation analysis, Format, Writing | Permalink | Comments (0)

Peter A. Todd of the National University of Singapore, and Richard J. Ladle of Oxford University, write in Correspondence this week (Nature 451, 244; 2008):
On 22 November, the Higher Education Funding Council for England announced that the assessment and funding of science-based disciplines will in future be "based on citation rates per paper, aggregated for each subject group at each institution".
Changes in performance indicators always strongly influence individual and institutional behaviour and 'citation game-playing' will no doubt become a staple of coffee-room conversation. What is less clear is how the citation practices of authors may influence bibliometric indicators.
Citation practices are known to be imperfect. The documented problems include excessive citation of an author's own work. Papers cited can be inappropriate or ambiguous in their support and, in some cases, the authors may not have read the papers they cite. Authors may form 'citation coalitions' within research networks. They may fail to provide citations to intellectual precursors or to work reporting conflicting conclusions. There are geographical and language biases. The increasing number of many-authored papers makes it impossible to have a clean-cut general metric in which one author is associated with one paper.
Taken together, these factors represent a problematic degree of error for the proposed bibliometric system of assessment. They place added responsibility on journal editors and reviewers as arbiters of appropriate author conduct.
Unfortunately, there are no simple solutions. Currently, identifying poor citation practices is not emphasized in the peer-review process, so perhaps journals could adopt a system of random citation audits, or periodically request evidence of citation appropriateness from authors. In reality, time constraints and the sheer volume of submissions to many journals mean that such measures are unlikely to be implemented soon.
Until referencing practices improve, we would argue that using citation rates to assess performance is fundamentally flawed.

Posted by Maxine Clarke on January 22, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

From this week's issue of Nature, a news story by Declan Butler "Free journal-ranking tool enters citation market" (Nature , 451, 6; 2008)
"A new Internet database lets users generate on-the-fly citation statistics of published research papers for free. The tool also calculates papers' impact factors using a new algorithm similar to PageRank, the algorithm Google uses to rank web pages. The open-access database is collaborating with Elsevier, the giant Amsterdam-based science publisher, and its underlying data come from Scopus, a subscription abstracts database created by Elsevier in 2004.
The SCImago Journal & Country Rank database was launched in December by SCImago, a data-mining and visualization group at the universities of Granada, Extremadura, Carlos III and Alcalá de Henares, all in Spain. It ranks journals and countries using such citation metrics as the popular, if controversial, Hirsch Index. It also includes a new metric: the SCImago Journal Rank (SJR)."
The article goes on to discuss the new metric, and to compare it with others, notably the dominant Impact Factor of Thomson Scientific. Thomson has set up a web forum to respond formally to an editorial in the Journal of Cell Biology, in which Mike Rossner and colleagues point to discrepancies between their "independent audit" calculations of Impact Factors based on Thomson data, and Thomson's own calculated Impact Factors from the same data.

Posted by Maxine Clarke on January 03, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Citation-based quality metrics were discussed on Nautilus earlier this year. One of those, the h (for highly cited) index, was covered recently in a News story, and is the subject of two Correspondence letters in the current issue of Nature.

Michael C. Wendel of Washington University Medical School writes (Nature 449, 403; 2007):
The h-index (the number n of a researcher's papers that have received at least n citations) may paint a more objective picture of productivity than some metrics, as your News story 'Achievement index climbs the ranks' (Nature 448, 737; 2007) points out. But for all such metrics, context is critical.
Many citations are used simply to flesh out a paper's introduction, having no real significance to the work. Citations are also sometimes made in a negative context, or to fraudulent or retracted publications. Other confounding factors include the practice of 'gratuitous authorship' and the so-called 'Matthew effect', whereby well-established researchers and projects are cited disproportionately more often than those that are less widely known. Finally, bibliometrics do not compensate for the well-known citation bias that favours review articles.

Clint D. Kelly and Michael D. Jennions of Australian National University, write (Nature 449, 403; 2007):
The h-index seems to be breaking away from the bibliometric pack, in the race to become a favoured measure of scientific performance ('Achievement index climbs the ranks' Nature 448, 737; 2007). However, if the h-index is to become an assessment tool commonly used by university administrators and government bureaucrats, those using it should be aware of its pitfalls.
As noted in your News story, tallying how many papers a researcher publishes (their productivity) gives undue merit to those who publish many inconsequential papers. But at least for ecologists and evolutionary biologists, the h-index is highly correlated with productivity.
This is worrisome, because the h-index is easily misconstrued as an equitable measure of research quality. We offer two examples.
First, female ecologists and evolutionary biologists publish fewer papers than their male counterparts, and they have significantly lower h-indices. Should administrators therefore conclude that men are better researchers? No. The gender difference vanishes if we control for productivity. It seems unlikely that this phenomenon is restricted to ecology and evolution.
Second, the h-index increases with age and using the ratio of the two can be problematic. Therefore, reliably comparing the performance of younger researchers with older ones is difficult.

Your views are welcome.

Posted by Maxine Clarke on October 04, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

Language Log: Citation Plagiarism?
From the Language Log entry linked above: "Plagiarism normally involves either the unacknowledged borrowing of someone else's idea or the unacknowledged borrowing of someone else's words. A third kind of plagiarism is, however, occasionally mentioned, namely the citation of a reference without acknowledging that it came from another source. If author Jones reads a paper by Smith and thereby learns of a paper by Doe and cites Doe without mentioning that he owes the reference to Smith, he has committed this kind of plagiarism, if plagiarism it be."
Bill Poser, author of the Language Log entry, goes on to argue why this type of plagarism is not, in his view, plagiarism, as there is no deception involved. The authors of the original reference may, in Dr Poser's view, "deserve more credit than they receive, but that is a different matter."

See here for the JISC plagiarism advisory service, which provides generic advice and guidance on all aspects of plagiarism prevention and detection to institutions, academics and students.

Posted by Maxine Clarke on August 22, 2007
Categories: Citation analysis, Ethics | Permalink | Comments (1)

What’s so wonderful about citations? asks Cambridge professor Peter Murray-Rust. Prof Murray-Rust has looked on Google Scholar for a paper which according to the publisher has more than 100,000 accesses, and found that it has 92 citations over the same period, which translates into one citation for every 1,000 (or so) downloads.
Prof Murray-Rust applied the same logic to himself. He was told by a publisher that his paper had been downloaded 6,000 times, so expected to find about 6 citations on Google Scholar -- but in the event found only one. "I’m not saying there are better ways - there probably aren’t", he writes. "If we make downloads a metric, then people will try to distort them. But let’s not take this [citation analysis] as seriously as we do."

Posted by Maxine Clarke on July 31, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

In a post entitled Copy-Editing and Citation-Linking , Michael Jubb of the Research Information Network compares the version of an article finalised by the author, and the version edited by the journal. An extract is provided here:

"Two recent articles in Learned Publishing, the journal published by the Association of Learned and Professional Society Publishers (ALPSP), have highlighted the issue. The first, by Wates and Campbell, looked at the changes made in copy-editing in articles published in a series of Blackwell journals. The second, by Goodman, Dowson and Yaremchuk, is in the current issue of Learned Publishing, but also, interestingly, through the University of Arizona’s repository. I have not tried to compare the two versions. It would be interesting to do so, not least because they found that as a result of publishers’ copy-editing “there were a number of differences between author-final and published versions that were ‘confusing’ and that sometimes the publisher version and sometimes the author version was the more confusing”.......
In an editorial ....Sally Morris also comments on the two articles, and lays considerable stress on the value that the publisher adds in the checking and formatting of references and the provision of citation linking via CrossRef..... the need to add DOI links is a relatively new one which I gather relatively few authors actually do themselves (and I was not guided so to do by the publishers of either of my recent articles)."
See here for the full article.
We would be interested to hear further feedback from authors about the editing and web services they received from Nature journals and NPG journals, to add to the regular "author experience" surveys we conduct.

Posted by Maxine Clarke on July 19, 2007
Categories: Author services, Citation analysis, Communication | Permalink | Comments (1)

Antonio G. Valdecasas and Uta Grothkopf write:
Leaving aside the adequacy of using citation counts to evaluate scientists' merit (as discussed for instance by Peter Lawrence in Nature 422, 259-261; 2003), there is the associated problem of exhaustive gathering of citations for certain kinds of publications, in particular monographs and book series that are not routinely covered by the Science Citation Index (SCI).
In countries like Spain where, as Lawrence points out, bureaucrats carry out scientists' assessment with "formulaic precision", there is, therefore, a risk of underrating this type of publication. Examples can be as extreme as to receive a dictum of almost 'no impact' for a monograph whose articles have been cited more than 150 times in journals included in the SCI, 19 times in non-SCI journals, 10 times in proceedings and which contains several articles in 'rare' languages such as Russian and Chinese. These citation results can be easily found through a simple search in Google Scholar. Not a bad citation number for a discipline (morphometrics) that has a mean number of citations for the period 1996-2005 of 8.5 (n= 1,408) citations per article, but most of them would be lost if one relies exclusively on the SCI.
Maybe the days of the SCI are numbered, as is already the case in disciplines such as astronomy, where alternative services are used. If impact is to be used as a metric that affects people directly, then databases like Google Scholar -- free, accessible to everybody, and non-discriminatory against languages other than English -- could provide a tool of universal coverage for bureaucrats and evaluation committees to discover the real impact of publications and hence to be less biased in the distribution of benefits.
Antonio G. Valdecasas
Museo Nacional de Ciencias Naturales, Madrid, Spain
Uta Grothkopf
European Southern Observatory, Garching, Germany.

Posted by Maxine Clarke on July 03, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (3)

The 2006 impact factors for the Nature journals can be viewed at the Author and Reviewers' website. Here is Nature Publishing Group's recent press release about the new (2006) Impact Factors.

In last week's Journal Citation Report (Thomson ISI), Nature Publishing Group (NPG) once again emerged as the high impact publisher. 12 of the top 30 titles are published by NPG. New titles have leapt to the top of their respective categories, and established titles retained their leading positions. Many of NPG's journals published on behalf of societies have also performed well.
NPG established a lead position in the physical science categories. Nature Physics, launched in October 2005, received its first impact factor of 12.04, securing its place as the leading monthly journal in physics. Nature Materials, now with its third impact factor, goes from strength to strength. Its impact factor leapt by 3.25 to 19.19, placing it first in three categories: materials, multidisciplinary; physics, applied; and chemistry, physical. Nature Chemical Biology also received its first impact factor of 12.41, ranking it in the top five monthly research journals in biochemistry and molecular biology.
Nature Methods has become the leading monthly journal in biochemical research methods, with its impact factor soaring from 6.74 to 14.96. NPG's open access journal, Molecular Systems Biology, received its first impact factor of 7.94. The seven journals from the Nature Reviews series, launched between 2001 and 2003, each rank as the leading monthly review journal in their specified category, significantly ahead of any competing review titles.
The Nature journals hold their leading positions across a range of categories. Nature Genetics and Nature Reviews Genetics hold the first and second positions in the category of genetics and heredity. Nature Medicine leads in medicine, research and experimental. Nature Reviews Immunology and Nature Immunology are the top monthly journals in immunology. Nature Reviews Neuroscience and Nature Neuroscience lead in the category of neuroscience. The category of cell biology is dominated by Nature journals - Nature Reviews Molecular Cell Biology (first), Nature Medicine (third), Nature Cell Biology (fifth) and Nature Structural & Molecular Biology (thirteenth).
NPG's academic and society journals also excelled, with most titles showing an increase in impact. The Journal of Investigative Dermatology secured an impact factor of 4.53 and is once again first among dermatology journals. Molecular Psychiatry ranks second in psychiatry with an impact factor of 11.8. Oncogene and Leukemia rank in the top 20 for oncology journals, and EMBO Journal and EMBO Reports also appear in the top 20 for cell biology journals. Cell Research's impact factor has risen to 3.4, making it the highest ranking journal in China and one of the highest in Asia. Cell Research is published on behalf of the Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences.
Nature retains its position as the most cited weekly science journal, with more than 390,000 citationa, an increase of almost 18,000 on last year's count. Nature continues to publish more articles than any other multidisciplinary journal.
"We are delighted with NPG's overall success this year, which sees the strengthening position of our new journals and the continued performance of our established titles. The authors that choose to publish with NPG are to be thanked for our strong results, says Annette Thomas, the company's Managing Director. NPG's service to societies remains strong, invigorating established titles with new content and new technology. We remain committed to high impact, innovative publishing."
This press release and all others can be viewed in the NPG press room.
The impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years. It is an independent measure calculated by Thomson/ISI (Institute for Scientific Information), Philadephia, USA.See the Author and Reviewers' website for more explanation of impact factors.

Posted by Maxine Clarke on July 02, 2007
Categories: Citation analysis, Publications | Permalink | Comments (1)

Scopus, the abstract and citation database of peer-reviewed literature and quality web sources, will incorporate the h-index soon. The h-index considers the publication records of an individual, the number of papers published over n years and the number of citations for each paper. The result is a single number, the h-index. To provide the user with additional clarity Scopus sys it will include visual aids that present a transparent overview of citation and publication patterns over time, revealing whether the h-index is dependent on a few highly cited papers or that the author’s papers have a relatively consistent volume of citations.
See here for a recent Nature Commentary on the h-index; and see here for a 2005 Nature news story "Index aims for fair ranking of scientists".

Posted by Maxine Clarke on May 01, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (2)

In the Editorial of the April issue of Nature Clinical Practice Rheumatology (3, 189; 2007), Editor in Chief Peter E. Lipsky writes: "The IF [Impact Factor] was envisioned over 50 years ago with the purpose of eliminating "the uncritical citation of fraudulent, incomplete or obsolete data by making it possible for the conscientious scholar to be aware of criticisms of earlier work" (Garfield E [1955] Science 122: 108–111). The IF has subsequently morphed into an institutionalized means of ranking the quality of scientific journals and, by implication, the individual articles published within them; for researchers, the IF influences employability, promotion, grant acceptability and bonus payments, and has been likened to a popularity contest."
Dr Lipsky discusses various flaws in the Impact Factor evaluation system, such as the lack of transparency of the formula by which the IF is calculated; that an erroneous but frequently cited article will bolster the ranking; disproportionate representation of review articles; and the differences in publication and citation frequencies in different disciplines. He asks whether his journal even actually wants an IF when it becomes eligible, concluding: "We do not feel that the current IF will reflect either our quality or our potential influence on clinical practice. Only if a new validated metric is developed that can evaluate the true quality and value of journal articles can we make real progress in improving the communication of new information in clinical medicine."
The full editorial is available at the journal's website or as a PDF here: Download file.
Comments are very welcome.

Posted by Maxine Clarke on April 27, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

From a Correspondence by Brian Haynes in Nature 446, 725; 2007.
ISI has recently delisted a number of publications from the Web of Science without informing the affected publishers or editors, or publishing a full list of the excisions. The motivation seems to have been to focus the Web of Science on journals and to move conference proceedings to another, little-known product, ISI Proceedings — notwithstanding the fact that many journals have special issues containing conference proceedings.
Proceedings of the Combustion Institute, an important archive in the multidisciplinary field of combustion dating back to 1928, is one of the affected publications. Because its peer-reviewed papers are presented at the biennial International Symposium on Combustion, they will no longer be listed in the Web of Science. According to ISI, the decision to exclude this publication "was not based on an evaluation of its importance to the community of scholars it serves".
This experience adds a new dimension to problems with excessive reliance on citation analyses. The Web of Science database itself is subject to unaccountable adjustments without scientific justification or regard to scientific importance.
Brian Haynes
The Combustion Institute, Pittsburgh, USA

Posted by Maxine Clarke on April 26, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

The United Kingdom Serials Group (UKSG), in association with the online usage metrics organisation COUNTER, is funding a study to explore how online journal usage statistics might form the basis of a new metric of journal quality, the "Usage Factor". The first stage of this project involved a series of interviews with various stakeholders, and the second, current stage involves a web-based survey designed to obtain the views of many more librarians and authors than was possible for the interview stage.

If you are an author of a publication in any of the Nature or Nature Publishing Group journals, we hope you can spare a few minutes to complete this important survey, which you can do by clicking on this link. The survey aims to:

* Discover what you think about the measures that are currently used to assess the value of scholarly journals (notably impact factors);
* Gauge the potential for usage-based measures;
* Provide an opportunity for you to suggest possible different, additional measures.

See here for the author survey. It will take less than 5 minutes to do; because it is due to close by 30 March (though it may be extended for a few days), please do visit the survey site now if you are interested in contributing your views and experience.

Posted by Maxine Clarke on March 28, 2007
Categories: Access, Citation analysis, Quality measures | Permalink | Comments (0)

Scopus, Elsevier’s abstract and citation database, has now added citation data, including abstracts, for many older papers, making it ever-closer to the rival Thomson’s Web of Science service. Nature Publishing Group titles are indexed by both databases, as well as by many other abstract&indexing services.

Since its launch two years ago, Scopus is now attracting publishers who want their older journals to be listed, including Springer, Nature Publishing Group, the American Physical Society, and the American Institute of Physics. Most of these publishers have digitized their content themselves back to the first issues of their journals. Older journals, such as Nature (first issue published on 4 November 1869, and freely available online), are still in the process of digitizing their very old content. Nature's digital archive currently goes back to 1950, with the issues back to 1869 to be added later this year. Nature New Biology and Nature Physical Sciences archives are also being digitized.

Scopus already contains 15 million pre-1996 records, and the new deal will add around seven million more. The new records will include full abstracts, traditionally a strength of Thomson’s Web of Science. Scopus does not necessarily intend to index older content in all areas, believing that 11 years of citation data is enough for some purposes - for example, to provide an overview of a publication for a grant or tenure application, so is prioritzing the digitization of older content accordingly.

In some cases, researchers may be interested in older content to see the citations that their own research has received over the years, or may use it to study historical trends. Whatever the needs of the reader, the addition of more search features and functions as the two organizations compete for customers can only be of benefit, especially taking into account features being introduced by the free-to-use service Google Scholar, also driving innovation and introduction of more reader services by the commercial providers.
See Nature Publishing Group abstracting and indexing page.

Posted by Maxine Clarke on January 26, 2007
Categories: Author services, Citation analysis, Publications | Permalink | Comments (0)

Citation analysis can loom large in a scientist's career. On pages 1003-1004 of the 21-28 December 2006 issue of Nature, Sune Lehmann, Andrew Jackson and Benny Lautrup compare commonly used measures of author quality. The mean number of citations per paper emerges as a better indicator than the more complex Hirsch index; a third method, the number of papers published per year, measures industry rather than ability. Careful citation analyses are useful, but Lehmann et al. caution that institutions often place too much faith in decisions reached by algorithm, use poor methodology or rely on inferior data sets.

Read the full text of the Commentary here (subscription or site licence required).

We welcome comments on this Commentary and on citation-based quality measures in general.

Posted by Maxine Clarke on January 04, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (6)