Tag Archives: ISI

IF all over again

Posted on by
Reply

If you can meet with Triumph and Disaster
And treat those two impostors just the same;
If you can bear to hear the truth you’ve spoken
Twisted by knaves to make a trap for fools,
Or watch the things you gave your life to, broken,
And stoop and build ’em up with worn-out tools:
Rudyard Kipling

Yes it is that time of the year again when Thomson Reuters publishes its Journal Citation Report (JCR) and everyone involved in science publishing gets obsessed with Impact Factors (IF). I’m not going to go through the arguments about how little Impact Factors really mean, and I’m certainly not going to try and forecast the health or otherwise of a publishing venture based on a change of 0.3 in its IF. But I thought you might want to know what Nature Protocols’s 2012 IF is. So cue drum roll …

It’s 7.96 down a couple of points from last year’s 9.92

Or

It’s 11.74 up from last year’s 10.20

Nothing is simple when it comes to Impact Factors. They are sort of an estimate of the average number of citations that a paper in a particular journal gets, but they are actually the number of citations a journal gets in a year to articles published in the previous two (or five) years divided by the number of articles published in those years that it seem appropriate to cite (‘simples!’). Herein lies the apparent contradiction in the numbers I gave above. Nature Protocol’s Impact Factor based on citations in 2012 to protocols published in 2010 and 2011–the two year impact factor (IF2)– is 7.96. The Impact factor based on citations in 2012 to protocols published between 2007 and 2011–the five year Impact Factor (IF5)–is 11.74.

For most journals there isn’t a whole lot of difference between the IF2 and the IF5, certainly less than 10% so when someone says Impact Factor they normally mean IF2. There are a few journals with big differences between the two values. The journal with the highest IF2 of all, CA: A Cancer Journal for Clinicians published by the American Cancer Society, has an IF2 of 153.46 and a IF5 of 88.55 which I interpret as meaning that what it publishes is extremely relevant for a couple of years (and so is highly cited) but after that it quickly loses its importance.

Conversely that Nature Protocols has a higher IF5 than IF2 could be taken as an indication that the protocols we publish remain relevant well beyond the first years after presentation. A measure that might bear that out would be the cited half-life of the journal. This is defined as the median age of the articles published in Nature Protocols that were cited in a given year (i.e. 2012). For Nature Protocols it is 4.9 (in 2011 it was 4.2), but that really doesn’t say a lot as Nature Protocols is a relatively young journal which has only been in existence since 2006. The maximum value for cited half-life we could have got would have been 6 and if there was no change in the rate of citation of our protocols over time a value of 3 would have been expected. We will need to be at least in our teens before I will put much store by cited half-life.

There is another confounding factor in all this for Nature Protocols and its name is DAVID.

In December 2008 we published online a protocol by Richard Lempicki and colleagues at National Cancer Institute at Frederick, Maryland called “Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources” (Nature Protocols 4, 44-57 doi:10.1038/nprot.2008.211 (2009)). It is our most cited paper having been cited more than 3,000 times. It is in fact the most highly cited paper from any Nature  journal, including Nature itself, published in 2009 (yes I know it was published in 2008 but it was in the January 2009 issue of the journal and so that makes it officially a 2009 paper). In 2012 alone it was cited upwards of 1,000 times. However since it was published in 2009 those citation do not contribute to our  IF2 although they do to our  IF5. It is difficult to get the figures to calculate the exact effect of a single paper on IF but a fair approximation would be to say that had those 1,000 citations been included in the calculation of our IF2 then it would have been a bit less than 3 points higher, while excluding them from our  IF5 would reduce that by about 0.8 making both values in the region of 10.9.

Yep, you’ve got it! Citations to a single paper seem to account for all the difference in our Impact Factors. Which simply shows again that IF may be a great poem, but it is a poor measure of the scientific literature.

 

The Season of the Impact Factor

Posted on by
Reply

It is that time of year again! A bit like Christmas, now is the time when journal editors wait expectantly to open their presents; or in this case present singular. I’m talking about the annual release of Impact Factors from ISI. I’m not going to go through all the reasons why Impact Factors may not be the best way to judge scientific research. Katharine made some comments about that last year and I haven’t changed my opinion much since I railed against their tyranny at another journal. But Impact Factors and the other citation metrics probably say something about a journal so I thought I’d give you the numbers for Nature Protocols and say what I’m taking from them.

A picture from the cover

My favourite cover image from 2011.

So the 2011 Impact Factor for Nature Protocols is 9.924 which I’m really happy about. It would be a presentable number for a journal publishing primary research, and we don’t publish primary research. I would expect protocols to be formally cited less often. Also it is up 1.562 points from the 2010 value so Nature Protocols is probably being cited more than 12 months ago and hopefully that is an indication that our articles are being used more than 12 months ago. If our articles are being used more than they were then that’s good news.

Impact Factors are quite good for making comparisons between similar journals, but sadly none of the other main publishers of protocols is indexed by ISI and so don’t get given Impact Factors. I analysed the citation of those journals a few months ago and I’ll let you draw your own conclusions about what thier IF might be if they were given one. What ISI does do is put Nature Protocols in a group of journals publishing “Biochemical Research Methods”. Our colleagues at Nature Methods top that list with an IF of 19.276. However it is a bit like comparing chalk and cheese as other journals in this group are things like Acta Crystallographica Section D: Biological Crystallography (IF 12.619), Current Opinion in Biotechnology (IF 7.711) and even PLoS Computational Biology (IF 5.215). Still in the group we rank third, up from fourth last year, so again pleasing.

The numbers that please me are those that suggest that Nature Protocols articles have some longevity. If you calculate IF over 5 years instead of 2 it rises to 10.201 suggesting that Protocols are relevant well beyond their initial publication. Many journals, possibly even most (I’m eyeballing rather than having done a proper analysis)  have a lower 5-year IF than their standard 2-year IF.

For a slightly more sophisticated measure there is the Eigen Factor score. This is calculated in a similar way to Google’s page rank algorithm in that not all links are of equal value and self citations aren’t counted at all.  For 2011 ours is 0.10716. I’m not completely clear how to interpret that but it is more than for 2010 so that must be good!

Last up the Article Influence Score. This is derived from the Eigen Factor and then normalised for the size of the journal so that the ‘average’ journal will have an Influence of 1.000. Ours for 2011 is 4.422 which is again more than it was last year and shows, to misquote the immortal Yogi, that we are “smarter than the average journal (Boo-Boo)”.

So it’s all good (for as much as Impact Factors and similar measures can tell you anything about a journal).  But at Nature Protocols we don’t really care about the value of our IF. In the next 12 months we will try to commission and publish even better protocols so that more scientists will be helped in their research. If someone can quantify that then I’d like to hear about it.