Nautilus

The 2007 Impact Factors are now out (published on 17 June 2008). The ten Nature Publishing Group journals with the highest Impact Factors are as follows:

1 NAT REV MOL CELL BIO 31.921
2 NAT REV CANCER 29.190
3 NATURE 28.751
4 NAT REV IMMUNOL 28.300
5 NAT MEDICINE 26.382
6 NAT IMMUNOLOGY 26.218
7 NAT GENETICS 25.556
8 NAT REV NEUROSCI 24.520
9 NAT REV DRUG DISCOV 23.308
10 NAT BIOTECHNOLOGY 22.848

The Impact Factors of the Nature journals that publish original research are:

1 NATURE 28.751
2 NAT MEDICINE 26.382
3 NAT IMMUNOLOGY 26.218
4 NAT GENETICS 25.556
5 NAT BIOTECHNOLOGY 22.848
6 NAT MATERIALS 19.782
7 NAT CELL BIOLOGY 17.623
8 NAT NEUROSCIENCE 15.664
9 NAT METHODS 15.478
10 NAT NANOTECHNOLOGY 14.917
11 NAT PHYSICS 14.677
12 NAT CHEM BIOLOGY 13.683
13 NAT STRUCT MOL BIOLOGY 11.085

(Nature Photonics and Nature Geoscience are not old enough to have been awarded an Impact Factor this year.)
Readers can create their own lists of journals by subject area, title, Impact Factor or publisher, at ISI Web of Knowledge.
There is a free-access account at the ThomsonISI website which explains how the Impact Factor for journals is calculated.
Discussion of the 2007 Impact Factors, and of citation in science in general, is taking place at the Nature Network Citation in Science group, which you are warmly invited to join.

Posted by Maxine Clarke on June 18, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

The June editorial in Nature Neuroscience (11, 619; 2008) discusses the relationship between web traffic and citations. The journal’s preliminary analysis indicates that the number of downloads a paper receives immediately following its appearance online correlates very well with its citation frequency years after publication. Noah Gray, one of the Nature Neuroscience editors, has written a post at Action Potential, the journal’s blog, to provide more of the details behind the data and analysis, and to initiate discussion. He writes (edited for length):
Everyone has their own pet problem with impact factors, but despite these concerns, these numbers are typically used to rate the importance or prominence of a particular journal, and thus by proxy, the importance of the individual papers published within. This is a seriously flawed use of association, leading scientists to often equate the total number of citations with scientific impact, which can be fraught with problems. Searching for an alternative measure of impact that is perhaps free of the “bias of authority” (citing a paper because it is from a famous lab) or the “lemming bias” (citing a paper just because everyone else seems to do so whenever broaching a particular subject) led us to explore readership….
The “number of downloads” measure potentially provides a piece of an alternative solution for deciphering the impact of an individual paper. In this current scientific climate where tenure and grant funding decisions are influenced by flawed metrics like impact factor, it is important to make good use of all available technology in an attempt to realize a better system of measuring the scientific impact of any particular paper. This analysis is obviously preliminary and flawed in its own ways, but perhaps metrics such as paper downloads can find a place in a compilation of aggregated stats, painting a more accurate and informative picture of manuscript influence.
The Nature Neuroscience editorial.
The Action Potential post and discussion.
Nature Network Citation in science forum discussion.
Nature Network Citation in science group homepage.
Futher reading: Connotea bookmarks "citation"
Further reading: Connotea bookmarks "impact factor" : thanks to Bob O'Hara for this library.

Posted by Maxine Clarke on June 05, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

In his Editorial 'Measuring success' in the April issue of EMBO Reports (9, 301; 2008), Frank Gannon looks at the pluses and minuses of metrics used to measure the success of scientists, their institutions and whole nations. He writes: "Notwithstanding the imperfection of the metrics, the resulting league tables are having real effects: university presidents world-wide await with trepidation the outcome of the latest scores. They know that it is easier to attract staff to a university that is moving up the ranking tables and this, inevitably, is leading to policy changes. Research areas that contribute little to the overall ranking might be closed and the appointments of new faculty members will reflect, to some extent, their potential to contribute to the university's metric success. Perversely, universities are entering a time of greater competition when co-operation might in fact be more appropriate. Governments also watch what is happening in the league tables, which translates into funding decisions. In this way, the power of the tables becomes amplified—although in keeping with the maxim quoted at the beginning of this article, such measurements will probably improve research in the long run because they stimulate competition. Often an external wake-up call is needed to end complacency and instigate much needed changes."

Posted by Maxine Clarke on April 03, 2008
Categories: Quality measures | Permalink | Comments (0)

Peter A. Todd of the National University of Singapore, and Richard J. Ladle of Oxford University, write in Correspondence this week (Nature 451, 244; 2008):
On 22 November, the Higher Education Funding Council for England announced that the assessment and funding of science-based disciplines will in future be "based on citation rates per paper, aggregated for each subject group at each institution".
Changes in performance indicators always strongly influence individual and institutional behaviour and 'citation game-playing' will no doubt become a staple of coffee-room conversation. What is less clear is how the citation practices of authors may influence bibliometric indicators.
Citation practices are known to be imperfect. The documented problems include excessive citation of an author's own work. Papers cited can be inappropriate or ambiguous in their support and, in some cases, the authors may not have read the papers they cite. Authors may form 'citation coalitions' within research networks. They may fail to provide citations to intellectual precursors or to work reporting conflicting conclusions. There are geographical and language biases. The increasing number of many-authored papers makes it impossible to have a clean-cut general metric in which one author is associated with one paper.
Taken together, these factors represent a problematic degree of error for the proposed bibliometric system of assessment. They place added responsibility on journal editors and reviewers as arbiters of appropriate author conduct.
Unfortunately, there are no simple solutions. Currently, identifying poor citation practices is not emphasized in the peer-review process, so perhaps journals could adopt a system of random citation audits, or periodically request evidence of citation appropriateness from authors. In reality, time constraints and the sheer volume of submissions to many journals mean that such measures are unlikely to be implemented soon.
Until referencing practices improve, we would argue that using citation rates to assess performance is fundamentally flawed.

Posted by Maxine Clarke on January 22, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

"Any issue of Nature today has nearly the same number of Articles and Letters as one from 1950, but about four times as many authors. The lone author has all but disappeared. In most fields outside mathematics, fewer and fewer people know enough to work and write alone. If they could, and could spare the time and effort to do so, their funding agencies and home institutions would not permit it." So writes Mott Greene of the University of Puget Sound in his recent (single-author, naturally) Nature essay "The demise of the lone author" (Nature 450, 1165; 2007).
Professor Greene goes on to discuss how this practice is affecting, and will affect, the system of awarding credit for work done, predicting that "in those fields where multiple authorship endangers the author credit system, we shall soon see institutionally initiated restriction on the number of authors. Paradoxically, this is likely to be endorsed by all parties as preferable to cinema-style specification of who actually did what. Most will prefer full credit for a few papers to little or no credit for many, considering where it matters most: university committees in charge of tenure, promotion and salary increments based on scholarly production. Given Nature's role in determining, as well as chronicling, how science is reported (see Nature 450, 1; 2007), interested parties could watch these pages to see whether a trend towards more restricted authorship is emerging."

Nature's policy on author contribution statements is here, and was introduced in an Editorial here.
Professor Greene's article is also available on the beautiful website that celebrates the history of the journal Nature.

Posted by Maxine Clarke on January 10, 2008
Categories: Authorship, Careers, Quality measures | Permalink | Comments (1)

From this week's issue of Nature, a news story by Declan Butler "Free journal-ranking tool enters citation market" (Nature , 451, 6; 2008)
"A new Internet database lets users generate on-the-fly citation statistics of published research papers for free. The tool also calculates papers' impact factors using a new algorithm similar to PageRank, the algorithm Google uses to rank web pages. The open-access database is collaborating with Elsevier, the giant Amsterdam-based science publisher, and its underlying data come from Scopus, a subscription abstracts database created by Elsevier in 2004.
The SCImago Journal & Country Rank database was launched in December by SCImago, a data-mining and visualization group at the universities of Granada, Extremadura, Carlos III and Alcalá de Henares, all in Spain. It ranks journals and countries using such citation metrics as the popular, if controversial, Hirsch Index. It also includes a new metric: the SCImago Journal Rank (SJR)."
The article goes on to discuss the new metric, and to compare it with others, notably the dominant Impact Factor of Thomson Scientific. Thomson has set up a web forum to respond formally to an editorial in the Journal of Cell Biology, in which Mike Rossner and colleagues point to discrepancies between their "independent audit" calculations of Impact Factors based on Thomson data, and Thomson's own calculated Impact Factors from the same data.

Posted by Maxine Clarke on January 03, 2008
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Juan-Carlos Lopez, via Spoonful of Medicine (Nature Medicine's blog), was chatting to some scientists at a conference, when one of them told him that some journals take advantage of the announcement of the Nobel prize to send out an e-mail highlighting the papers by the laureates that they have had the privilege to publish. Juan-Carlos writes: "I don't know about you but such a marketing strategy strikes me as somewhat cheeky...... if other publishing firms are currently entertaining a similar strategy, here's an idea to turn it on its head -- send e-mails highlighting the papers from the laureates that your journal has REJECTED and the name of the publication where they were ultimately published.......It's a shame that confidentiality issues get in the way of such an idea because, if you were to send such an e-mail, people would surely be talking about your journal."

Posted by Maxine Clarke on October 23, 2007
Categories: Awards, Quality measures | Permalink | Comments (0)

Philip Esler, chief executive of the Arts and Humanities Research Council, Bristol, UK, writes in Nature's Correspondence page this week (Nature 449, 781; 2007):
The United Kingdom's research base has seen unprecedented increases in public investment in recent years, mostly predicated on the long-term benefits to society expected to arise from that investment. It is the research councils' responsibility, as the major public funders of UK research, to provide compelling evidence that these expectations are being met. Your Editorial 'Innovation versus science?' (Nature 448, 839; 2007) concludes that efforts to document this herald a shift away from our support for basic research. As a research council chief executive, leading our efforts to increase our economic impact, I can say that is not the case.
The UK Research Councils have just published a report, Excellence with Impact, that looks across research councils' investments. Each of 18 case studies shows actual and/or potential impact, ranging from biotech spin-outs and skilled engineers to climate-change policy. Probably the most reassuring finding was the extent to which some demonstrated multiple types of impact. Furthermore, many of the impacts were not necessarily part of the original rationale for the specific investment, suggesting that serendipity and opportunism are important factors for the research councils. Investment in DNA technologies, for example, did not anticipate the forensic power of DNA fingerprinting, and polymer research was not funded with the anticipation that it would create a new market in flexible displays.
These results demonstrate the wisdom of the research councils' commitment to funding excellent basic research. Rather than weaken that commitment, our approach is to embed economic-impact considerations in our organizations, thus shifting the central focus of the research councils to excellent research with high economic impact. So it is about what basic research we should fund, rather than if we should fund it.

Posted by Maxine Clarke on October 22, 2007
Categories: Quality measures | Permalink | Comments (0)

Outsourcing has been relatively rare in academia until now, but is that beginning to change? So ask Franz B. Pichler and Susan J. Turner of the University of Aukland in their Commentary in this month's Nature Biotechnology (25, 1093-1096; 2007). They write: "The rapid development of ever more complex and expensive technology coupled with the increasingly competitive environment in the life sciences is changing both how we access technology and how we conduct research. It is no longer possible to expect every technology to be readily available within a research institution, let alone a laboratory, yet access to such technology is often the difference between success and failure within today's competitive funding models. To fully embrace emerging technologies, scientists are increasingly reliant on outsourcing to contract technology providers (CTPs). In this context, CTPs are companies or institutes that conduct partial or entire experiments on a commercial basis."
The Commentary addresses the pros and cons of going out of house, and some of the strategies needed to make sure this novel form of collaboration works. On what side of the fence sit the authors? "Ultimately, science is more about the conceptualization of the experiment, its design, analysis and interpretation than the actuality of conducting an experiment. Provided that the experiment is performed to the required specifications, it should not matter that some or all of the work has been outsourced. As outsourcing can achieve significant efficiencies in research, we predict that it will become an increasingly common component of research programs, even in academia."

Posted by Maxine Clarke on October 12, 2007
Categories: Careers, Partners, Quality measures | Permalink | Comments (0)

"What would you do if you could publish only 20 papers throughout your career?" asks Juan-Carlos Lopez on Spoonful of Medicine blog, referring to this month's Nature Medicine editorial (free access at Nat. Med. 13, 1121; 2007). If scientists would agree to limit their output to 20 papers over a career, would this clean up the scientific literature and improve on the peer-review process? Juan-Carlos writes: "many articles reporting incremental advances would no longer be written, and many specialized journals would disappear. And with far fewer papers to read, each one reporting a much more complete piece of research, search committees or funding bodies could directly evaluate the work of a given scientist, instead of leaning on surrogate indicators such as a journal's impact factor or number of citations, "evil" numbers that many researchers love to hate."
See the Nature Medicine editorial for further details of this radical proposal.

Posted by Maxine Clarke on October 11, 2007
Categories: Publishing, Quality measures | Permalink | Comments (0)

Citation-based quality metrics were discussed on Nautilus earlier this year. One of those, the h (for highly cited) index, was covered recently in a News story, and is the subject of two Correspondence letters in the current issue of Nature.

Michael C. Wendel of Washington University Medical School writes (Nature 449, 403; 2007):
The h-index (the number n of a researcher's papers that have received at least n citations) may paint a more objective picture of productivity than some metrics, as your News story 'Achievement index climbs the ranks' (Nature 448, 737; 2007) points out. But for all such metrics, context is critical.
Many citations are used simply to flesh out a paper's introduction, having no real significance to the work. Citations are also sometimes made in a negative context, or to fraudulent or retracted publications. Other confounding factors include the practice of 'gratuitous authorship' and the so-called 'Matthew effect', whereby well-established researchers and projects are cited disproportionately more often than those that are less widely known. Finally, bibliometrics do not compensate for the well-known citation bias that favours review articles.

Clint D. Kelly and Michael D. Jennions of Australian National University, write (Nature 449, 403; 2007):
The h-index seems to be breaking away from the bibliometric pack, in the race to become a favoured measure of scientific performance ('Achievement index climbs the ranks' Nature 448, 737; 2007). However, if the h-index is to become an assessment tool commonly used by university administrators and government bureaucrats, those using it should be aware of its pitfalls.
As noted in your News story, tallying how many papers a researcher publishes (their productivity) gives undue merit to those who publish many inconsequential papers. But at least for ecologists and evolutionary biologists, the h-index is highly correlated with productivity.
This is worrisome, because the h-index is easily misconstrued as an equitable measure of research quality. We offer two examples.
First, female ecologists and evolutionary biologists publish fewer papers than their male counterparts, and they have significantly lower h-indices. Should administrators therefore conclude that men are better researchers? No. The gender difference vanishes if we control for productivity. It seems unlikely that this phenomenon is restricted to ecology and evolution.
Second, the h-index increases with age and using the ratio of the two can be problematic. Therefore, reliably comparing the performance of younger researchers with older ones is difficult.

Your views are welcome.

Posted by Maxine Clarke on October 04, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

Funding woes plague US biomedical researchers. But calls for more funding ignore the structural problems that push universities to produce too many scientists, argues Brian C. Martinson in a Nature Commentary this week (Nature 449, 141-142; 2007). Instead, existing researchers should be given more time, space and freedom to ask questions in new ways, to take risks, and to innovate. Reducing the intensity of competition for NIH (National Institutes of Health) funds is one way of making this happen. Dr Martinson writes:

There are two main routes to contraction of the academic workforce today — through tenure failures, and with younger investigators shifting from academia into industry research. This is worrisome for university research in particular because history suggests that the most dramatic innovations come from the young. So is the only solution to force long-time NIH grant getters into retirement? Perhaps not. Universities have benefited handsomely from the efforts of senior faculty members in securing NIH grants during their careers, perhaps those same universities could now return the favour by taking full responsibility for paying these faculty salaries in their later years. This would serve the dual purpose of getting them off the NIH dole, and encouraging them to share their knowledge with their younger colleagues through more teaching.
This won't be easy. Given the levels of dependency on NIH money, it is akin to asking an addict to give up an easy fix. And not all universities will be in financial positions to employ this strategy, but it's difficult to imagine that richer institutions — some of whom acknowledge that their success lies in capturing an increasing share of the NIH pie— could not lead the way in this. Prospective students and their parents may also look favourably on senior faculty members spending more time teaching.

See here for the full Commentary article.
A related Editorial on the NIH grant-allocation process is discussed at Peer to Peer.

Posted by Maxine Clarke on September 14, 2007
Categories: Careers, Quality measures | Permalink | Comments (1)

What’s so wonderful about citations? asks Cambridge professor Peter Murray-Rust. Prof Murray-Rust has looked on Google Scholar for a paper which according to the publisher has more than 100,000 accesses, and found that it has 92 citations over the same period, which translates into one citation for every 1,000 (or so) downloads.
Prof Murray-Rust applied the same logic to himself. He was told by a publisher that his paper had been downloaded 6,000 times, so expected to find about 6 citations on Google Scholar -- but in the event found only one. "I’m not saying there are better ways - there probably aren’t", he writes. "If we make downloads a metric, then people will try to distort them. But let’s not take this [citation analysis] as seriously as we do."

Posted by Maxine Clarke on July 31, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

Is Citation Extortion practised? asks Peter Murray-Rust, a professor of chemistry at Cambridge University. He writes of a researcher he met who said she had submitted a manuscript and been told by the publisher (or editor) that it would not be published unless she included at least two citations to papers published by that publisher. Professor Murray-Rust is understandably appalled by this report, and asks readers of his blog whether this is a routine experience for them when submitting papers. Certainly this practice does not occur at any journals published by Nature Publishing Group: the citation list is entirely up to the author, although peer-reviewers (who are independent of the journals and of each other) might make suggestions as part of the revision process.

Posted by Maxine Clarke on July 09, 2007
Categories: Policies Feedback, Quality measures | Permalink | Comments (0)

12.040 : Article : Nature Physics

"12.040. Of course we're pleased. We want to shout it from the rooftops. Nature Physics has its first impact factor, and its a good one." Read more at the link above, this month's Nature Physics Editorial.

See here for the Nature Publishing Group press release about its journals' 2006 impact factors.
See the author and reviewers' website for a list of Nature journals and their impact factors.

Posted by Maxine Clarke on July 03, 2007
Categories: Publications, Quality measures | Permalink | Comments (0)

Antonio G. Valdecasas and Uta Grothkopf write:
Leaving aside the adequacy of using citation counts to evaluate scientists' merit (as discussed for instance by Peter Lawrence in Nature 422, 259-261; 2003), there is the associated problem of exhaustive gathering of citations for certain kinds of publications, in particular monographs and book series that are not routinely covered by the Science Citation Index (SCI).
In countries like Spain where, as Lawrence points out, bureaucrats carry out scientists' assessment with "formulaic precision", there is, therefore, a risk of underrating this type of publication. Examples can be as extreme as to receive a dictum of almost 'no impact' for a monograph whose articles have been cited more than 150 times in journals included in the SCI, 19 times in non-SCI journals, 10 times in proceedings and which contains several articles in 'rare' languages such as Russian and Chinese. These citation results can be easily found through a simple search in Google Scholar. Not a bad citation number for a discipline (morphometrics) that has a mean number of citations for the period 1996-2005 of 8.5 (n= 1,408) citations per article, but most of them would be lost if one relies exclusively on the SCI.
Maybe the days of the SCI are numbered, as is already the case in disciplines such as astronomy, where alternative services are used. If impact is to be used as a metric that affects people directly, then databases like Google Scholar -- free, accessible to everybody, and non-discriminatory against languages other than English -- could provide a tool of universal coverage for bureaucrats and evaluation committees to discover the real impact of publications and hence to be less biased in the distribution of benefits.
Antonio G. Valdecasas
Museo Nacional de Ciencias Naturales, Madrid, Spain
Uta Grothkopf
European Southern Observatory, Garching, Germany.

Posted by Maxine Clarke on July 03, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (3)

In this post on Nascent, NPG's web publishing and science blog, Timo Hannay provides a draft of his recent article in STM news : Nascent: Foo and beyond. The whole section is worth reading (and there is a good graphic of the opportunities provided by the "scientific web"), but here is an excerpt:

"The idea that everyone can now do their own publishing, making publishers superfluous, is misguided. But publishers do need to adapt. Online communities don't just happen, they require initiators, motivators, organisers, moderators, summarisers and guides. They also need trust systems based on user identification and reputation. In many ways, these, too, are traditional publishing roles, but they require new skills. Writers and editors now need to double as moderators and hosts. Publishers need to become adept at mitigating gaming and spamming of their systems, and at monetizing web traffic rather than selling subscriptions. On top of that, they need to become better at cooperating — with each other and with other organisations outside the industry. This particularly applies to online interoperability (even horror of horrors, with competitors), which is a positive-sum game that can benefit all participants. CrossRef has blazed a trail in this area, and we should build in its success.
Above all, publishers need to be leading the online charge, not following the scientists we serve. We are the information dissemination experts, so if we aren't pushing the boundaries and testing what's possible in this new world then we're not merely missing out, we're also not doing our jobs. Cynics will point out that most apparent 'opportunities' are a long way from turning a profit, and many probably never will. They're right. Do any of the STM projects I've mentioned above make a lot of money? No. But are they representative of the future of scientific communication, and do they provide a platform on which to build information businesses of the future? You'd better believe it."

In a similar vein, Alex Palazzo of The Daily Transcript blog wrote about Nature Publishing Group's "game plan" as he calls it, regarding science publishing and web "2.0" (the social, interactive web). The post arose from Alex's attendance at Nature Network Boston's pub night. This post, and the lively set of comments accompanying it, range over the the topic of the value of publishing in a journal, "open access" publication, and whether the unit of publication will become the paper itself rather than the journal in which it is published.

Posted by Maxine Clarke on July 02, 2007
Categories: Access, Communication, Publishing, Quality measures | Permalink | Comments (0)

Via news @ nature.com

Computer scientists at Columbia University in New York have used a mathematical model to estimate the number of flawed scientific papers that go unretracted, and its relation to journal impact factors.In correspondence published in EMBO Reports (M. Cokol et al. EMBO Rep. 8, 5, 422–423; 2007), the researchers find that fewer papers are retracted by journals with low impact factors. But their model raises as many questions as it answers, say specialists in scientific publishing, some of whom argue that it greatly oversimplifies the issues.

From the Nature story: "scientists and editors familiar with retraction issues are sceptical of the quality of the model's input data. Theoretical modelling exercises will generate bad results if the input data are flawed, says Drummond Rennie, deputy editor of the Journal of the American Medical Association, and a medical researcher at the Institute for Health Policy Studies, at the University of California, San Francisco.
Although the number of retracted articles is probably only the tip of the iceberg in terms of the number that should have been retracted, the model — based on journal impact factor and number of retractions — is too simplistic to capture the complex reality of the issues affecting the size and nature of the hidden part, Rennie says."

The full news @ nature.com story is here (site licence or subscription required).

Posted by Maxine Clarke on June 05, 2007
Categories: Ethics, Quality measures | Permalink | Comments (1)

Scopus, the abstract and citation database of peer-reviewed literature and quality web sources, will incorporate the h-index soon. The h-index considers the publication records of an individual, the number of papers published over n years and the number of citations for each paper. The result is a single number, the h-index. To provide the user with additional clarity Scopus sys it will include visual aids that present a transparent overview of citation and publication patterns over time, revealing whether the h-index is dependent on a few highly cited papers or that the author’s papers have a relatively consistent volume of citations.
See here for a recent Nature Commentary on the h-index; and see here for a 2005 Nature news story "Index aims for fair ranking of scientists".

Posted by Maxine Clarke on May 01, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (2)

In the Editorial of the April issue of Nature Clinical Practice Rheumatology (3, 189; 2007), Editor in Chief Peter E. Lipsky writes: "The IF [Impact Factor] was envisioned over 50 years ago with the purpose of eliminating "the uncritical citation of fraudulent, incomplete or obsolete data by making it possible for the conscientious scholar to be aware of criticisms of earlier work" (Garfield E [1955] Science 122: 108–111). The IF has subsequently morphed into an institutionalized means of ranking the quality of scientific journals and, by implication, the individual articles published within them; for researchers, the IF influences employability, promotion, grant acceptability and bonus payments, and has been likened to a popularity contest."
Dr Lipsky discusses various flaws in the Impact Factor evaluation system, such as the lack of transparency of the formula by which the IF is calculated; that an erroneous but frequently cited article will bolster the ranking; disproportionate representation of review articles; and the differences in publication and citation frequencies in different disciplines. He asks whether his journal even actually wants an IF when it becomes eligible, concluding: "We do not feel that the current IF will reflect either our quality or our potential influence on clinical practice. Only if a new validated metric is developed that can evaluate the true quality and value of journal articles can we make real progress in improving the communication of new information in clinical medicine."
The full editorial is available at the journal's website or as a PDF here: Download file.
Comments are very welcome.

Posted by Maxine Clarke on April 27, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (0)

From a Correspondence by Brian Haynes in Nature 446, 725; 2007.
ISI has recently delisted a number of publications from the Web of Science without informing the affected publishers or editors, or publishing a full list of the excisions. The motivation seems to have been to focus the Web of Science on journals and to move conference proceedings to another, little-known product, ISI Proceedings — notwithstanding the fact that many journals have special issues containing conference proceedings.
Proceedings of the Combustion Institute, an important archive in the multidisciplinary field of combustion dating back to 1928, is one of the affected publications. Because its peer-reviewed papers are presented at the biennial International Symposium on Combustion, they will no longer be listed in the Web of Science. According to ISI, the decision to exclude this publication "was not based on an evaluation of its importance to the community of scholars it serves".
This experience adds a new dimension to problems with excessive reliance on citation analyses. The Web of Science database itself is subject to unaccountable adjustments without scientific justification or regard to scientific importance.
Brian Haynes
The Combustion Institute, Pittsburgh, USA

Posted by Maxine Clarke on April 26, 2007
Categories: Citation analysis, Quality measures | Permalink | Comments (1)

The United Kingdom Serials Group (UKSG), in association with the online usage metrics organisation COUNTER, is funding a study to explore how online journal usage statistics might form the basis of a new metric of journal quality, the "Usage Factor". The first stage of this project involved a series of interviews with various stakeholders, and the second, current stage involves a web-based survey designed to obtain the views of many more librarians and authors than was possible for the interview stage.

If you are an author of a publication in any of the Nature or Nature Publishing Group journals, we hope you can spare a few minutes to complete this important survey, which you can do by clicking on this link. The survey aims to:

* Discover what you think about the measures that are currently used to assess the value of scholarly journals (notably impact factors);
* Gauge the potential for usage-based measures;
* Provide an opportunity for you to suggest possible different, additional measures.

See here for the author survey. It will take less than 5 minutes to do; because it is due to close by 30 March (though it may be extended for a few days), please do visit the survey site now if you are interested in contributing your views and experience.

Posted by Maxine Clarke on March 28, 2007
Categories: Access, Citation analysis, Quality measures | Permalink | Comments (0)

I'm posting here a comment made by Thomas Doering to the earlier Nautilus entry "Measures for Measures":

"As far as I understand only the shortfalls of visibility measures (i.e. various citation indices) have been discussed on this blog so far. And indeed, the list of drawbacks of citation-based measures can easily be extended (see for example Adam, D. 2002. Nature 415: 726-729; Kurmis, A. P. 2003. J. Bone Joint Surg. 85: 2449-2454). Why, however, do we concentrate so much on discussing visibility at all? Since in many cases visibility is a rather poor surrogate for assessing research quality, why don’t we just ask scientists to give their opinion on the quality of a paper directly? For example, the webpage www.CiteUlike.org allows publicly viewable rating of a paper and posting comments on it – although there, rating is only based on one parameter, reading priority. This platform could, however, relatively easily be developed into a powerful quality assessment tool, e.g. by adding a few more rating questions"

There are other websites in addition to CiteULike for this type of measure, for example Nature Publishing Group's free resource Connotea, for organizing, tagging, sharing and ranking articles. I've made some author-related Connotea tags for readers of this blog (see left-hand vertical navigation bar), but the primary usefulness of Connotea is for sharing and ranking articles published in the scientific literature. As well as ranking based on "number of users who add the article to their library", as mentioned by Dr Doering, Connotea has a note function to allow the user to add customised comments on the selected article. I agree it would be fascinating to encourage widespread use of these resources among scientists, to give a more qualitiative view of the literature than systems that use a "numbers-only" approach.

Posted by Maxine Clarke on March 02, 2007
Categories: Publications, Quality measures | Permalink | Comments (1)

In his Nature Correspondence "Time to give due weight to the 'carbon footprint' issue", Geoffrey Hammond writes: "The media are increasingly using the term 'carbon footprint' in articles about the need to mitigate climate change by reducing our carbon dioxide emissions. Footprints are spatial indicators, measured in hectares or square metres. The property that is often referred to as a carbon footprint is actually a 'carbon weight' of kilograms or tonnes per person or activity.
To improve public understanding of the issues surrounding climate change, it is necessary to be precise. Other 'footprints', such as the ecological or environmental footprint, convert resource consumption and waste production into spatial units. The term 'ecological footprint' was coined by William E. Rees, a planner at the University of British Columbia — who had previously used the term 'appropriated carrying capacity' — after a computer delivery man told him that the new machine, which took up less space than his old model, had a 'smaller footprint'.
As well as the media, many government agencies and environmental groups now use the expression 'carbon footprint'. Those who favour precision in such matters should perhaps campaign for it to be called 'carbon weight', or some similar term. That would avoid lasting confusion. Losing weight might even take on a whole new meaning."
Nature 445, 256 (18 January 2007) | doi:10.1038/445256b; Published online 17 January 2007.
We welcome suggestions from readers.

Posted by Maxine Clarke on January 31, 2007
Categories: Quality measures, Writing | Permalink | Comments (0)

Rebels hold their own in journal price war : Article : Nature (Subscription or site licence required.)

In a News story in the current issue of Nature, "Rebels hold their own in journal price war", Declan Butler looks at what has happened since last August, when the entire editorial board of the Elsevier journal Topology quit in a row over pricing. Now the board is setting up a non-profit competitor to be published by the London Mathematical Society. The Journal of Topology, announced last week, will launch next January and will cost US$570 per year, compared with Topology's $1,665.

Over the past eight years, continues the News story, around a dozen cheap or open-access journals have been created to compete directly with an expensive commercial journal, many by editorial boards that had quit the original publication in protest. (See the News story for a table of these journals.) But do the cheaper journals fare better than their rivals?

As far as scientific credibility is concerned, the answer is often yes — many of the challengers have obtained impact factors (a measure of the citations its papers receive) higher than their competitor. Nevertheless, the rebel journals often get poor support from libraries, with subscriptions being an uphill battle for them.

Source: Nature 445, 351 (25 January 2007) | doi:10.1038/445351a; Published onlin