Tag Archives: nature protocols

IF all over again

Posted on by

If you can meet with Triumph and Disaster
And treat those two impostors just the same;
If you can bear to hear the truth you’ve spoken
Twisted by knaves to make a trap for fools,
Or watch the things you gave your life to, broken,
And stoop and build ’em up with worn-out tools:
Rudyard Kipling

Yes it is that time of the year again when Thomson Reuters publishes its Journal Citation Report (JCR) and everyone involved in science publishing gets obsessed with Impact Factors (IF). I’m not going to go through the arguments about how little Impact Factors really mean, and I’m certainly not going to try and forecast the health or otherwise of a publishing venture based on a change of 0.3 in its IF. But I thought you might want to know what Nature Protocols’s 2012 IF is. So cue drum roll …

It’s 7.96 down a couple of points from last year’s 9.92

Or

It’s 11.74 up from last year’s 10.20

Nothing is simple when it comes to Impact Factors. They are sort of an estimate of the average number of citations that a paper in a particular journal gets, but they are actually the number of citations a journal gets in a year to articles published in the previous two (or five) years divided by the number of articles published in those years that it seem appropriate to cite (‘simples!’). Herein lies the apparent contradiction in the numbers I gave above. Nature Protocol’s Impact Factor based on citations in 2012 to protocols published in 2010 and 2011–the two year impact factor (IF2)– is 7.96. The Impact factor based on citations in 2012 to protocols published between 2007 and 2011–the five year Impact Factor (IF5)–is 11.74.

For most journals there isn’t a whole lot of difference between the IF2 and the IF5, certainly less than 10% so when someone says Impact Factor they normally mean IF2. There are a few journals with big differences between the two values. The journal with the highest IF2 of all, CA: A Cancer Journal for Clinicians published by the American Cancer Society, has an IF2 of 153.46 and a IF5 of 88.55 which I interpret as meaning that what it publishes is extremely relevant for a couple of years (and so is highly cited) but after that it quickly loses its importance.

Conversely that Nature Protocols has a higher IF5 than IF2 could be taken as an indication that the protocols we publish remain relevant well beyond the first years after presentation. A measure that might bear that out would be the cited half-life of the journal. This is defined as the median age of the articles published in Nature Protocols that were cited in a given year (i.e. 2012). For Nature Protocols it is 4.9 (in 2011 it was 4.2), but that really doesn’t say a lot as Nature Protocols is a relatively young journal which has only been in existence since 2006. The maximum value for cited half-life we could have got would have been 6 and if there was no change in the rate of citation of our protocols over time a value of 3 would have been expected. We will need to be at least in our teens before I will put much store by cited half-life.

There is another confounding factor in all this for Nature Protocols and its name is DAVID.

In December 2008 we published online a protocol by Richard Lempicki and colleagues at National Cancer Institute at Frederick, Maryland called “Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources” (Nature Protocols 4, 44-57 doi:10.1038/nprot.2008.211 (2009)). It is our most cited paper having been cited more than 3,000 times. It is in fact the most highly cited paper from any Nature  journal, including Nature itself, published in 2009 (yes I know it was published in 2008 but it was in the January 2009 issue of the journal and so that makes it officially a 2009 paper). In 2012 alone it was cited upwards of 1,000 times. However since it was published in 2009 those citation do not contribute to our  IF2 although they do to our  IF5. It is difficult to get the figures to calculate the exact effect of a single paper on IF but a fair approximation would be to say that had those 1,000 citations been included in the calculation of our IF2 then it would have been a bit less than 3 points higher, while excluding them from our  IF5 would reduce that by about 0.8 making both values in the region of 10.9.

Yep, you’ve got it! Citations to a single paper seem to account for all the difference in our Impact Factors. Which simply shows again that IF may be a great poem, but it is a poor measure of the scientific literature.

 

Protocols Discussion Forum is Moving

Posted on by

As you may or may not know Nature Protocols has a Discussion Forum where anyone having problems with their experiments can ask questions and hopefully receive helpful advice from other researchers with some experience in whatever is causing the problems. For the last several years this has been hosted on Nature Network but as of, well right about now, we are shifting its platform to a Google Group.

We have always thought of Nature Protocols as more than a journal and we try to engage with our readers to help them in their research. We have done this through commenting on our manuscripts; this blog; our YouTube channel and the discussion forum. We are always on the look out for technologies that might be helpful.

That’s why we are now moving the Discussion Forum onto the Google Groups platform.  We hope that this will greatly improve the effectiveness of the discussions by making it even easier for anyone to become involved in the forum. It should be easier to access on mobile devices, and easier to post on as comments can be made and discussion topics started simply by sending an email to nature-protocols-discussion@googlegroups.com.There are other technical advantages for us as well, not least a better spam filtering system so hopefully far fewer topics or posts will be offering fake passports for sale!

There are two ways to access the new look discussion forum, either through the Discussion page on this blog or directly at Google Groups . We very much hope that this move will increase the usefulness of the Discussion Forum by making it easier for researchers to take part, easier for the topics to be found by people searching for answers using search engines, and easier for topics to be connected to from throughout the Web.

The Current Discussion Forum isn’t going away but will remain as an archive. However it will not be possible to post new topics or reply to existing threads. I have however copied across all discussions started this year into the new platform as well as some of the most active from 2012. If there are any threads that you would like to see continued please let me know and I will copy those across as well.

I hope that everyone will find the new forum at least as helpful as it’s predecessor.

So, what shall we talk about? As an eponymous fictional talk show psychiatrist didn’t quite say:

“We’re listening.”

 

The Season of the Impact Factor

Posted on by

It is that time of year again! A bit like Christmas, now is the time when journal editors wait expectantly to open their presents; or in this case present singular. I’m talking about the annual release of Impact Factors from ISI. I’m not going to go through all the reasons why Impact Factors may not be the best way to judge scientific research. Katharine made some comments about that last year and I haven’t changed my opinion much since I railed against their tyranny at another journal. But Impact Factors and the other citation metrics probably say something about a journal so I thought I’d give you the numbers for Nature Protocols and say what I’m taking from them.

A picture from the cover

My favourite cover image from 2011.

So the 2011 Impact Factor for Nature Protocols is 9.924 which I’m really happy about. It would be a presentable number for a journal publishing primary research, and we don’t publish primary research. I would expect protocols to be formally cited less often. Also it is up 1.562 points from the 2010 value so Nature Protocols is probably being cited more than 12 months ago and hopefully that is an indication that our articles are being used more than 12 months ago. If our articles are being used more than they were then that’s good news.

Impact Factors are quite good for making comparisons between similar journals, but sadly none of the other main publishers of protocols is indexed by ISI and so don’t get given Impact Factors. I analysed the citation of those journals a few months ago and I’ll let you draw your own conclusions about what thier IF might be if they were given one. What ISI does do is put Nature Protocols in a group of journals publishing “Biochemical Research Methods”. Our colleagues at Nature Methods top that list with an IF of 19.276. However it is a bit like comparing chalk and cheese as other journals in this group are things like Acta Crystallographica Section D: Biological Crystallography (IF 12.619), Current Opinion in Biotechnology (IF 7.711) and even PLoS Computational Biology (IF 5.215). Still in the group we rank third, up from fourth last year, so again pleasing.

The numbers that please me are those that suggest that Nature Protocols articles have some longevity. If you calculate IF over 5 years instead of 2 it rises to 10.201 suggesting that Protocols are relevant well beyond their initial publication. Many journals, possibly even most (I’m eyeballing rather than having done a proper analysis)  have a lower 5-year IF than their standard 2-year IF.

For a slightly more sophisticated measure there is the Eigen Factor score. This is calculated in a similar way to Google’s page rank algorithm in that not all links are of equal value and self citations aren’t counted at all.  For 2011 ours is 0.10716. I’m not completely clear how to interpret that but it is more than for 2010 so that must be good!

Last up the Article Influence Score. This is derived from the Eigen Factor and then normalised for the size of the journal so that the ‘average’ journal will have an Influence of 1.000. Ours for 2011 is 4.422 which is again more than it was last year and shows, to misquote the immortal Yogi, that we are “smarter than the average journal (Boo-Boo)”.

So it’s all good (for as much as Impact Factors and similar measures can tell you anything about a journal).  But at Nature Protocols we don’t really care about the value of our IF. In the next 12 months we will try to commission and publish even better protocols so that more scientists will be helped in their research. If someone can quantify that then I’d like to hear about it.

Nature Protocols and Impact Factors

Posted on by

When we first launched "Nature Protocols ":https://www.nature.com/nprot I was keen to get our protocols indexed by all the big players. Whilst there was going to be a marketing campaign with ads in Nature and banner ads all over nature.com I knew we’d only get real users when people did their literature search and one of our protocols came top of the list. We wanted to be something researchers needed and found useful. However, alongside getting listed in the ISI Web of Knowledge came the news we’d also be eligible for an Impact Factor. It was clearly something scientists cared about. A lot of our early correspondence was about what our Impact Factor was. We had to be very apologetic and explain that we didn’t have one – first we’d need to publish some protocols, and then wait to see if they were cited. Personally I was relieved we didn’t have one because I wasn’t sure what an impact factor would mean for us, even if we did have one.

The Impact Factor comes into its own as a comparative factor. That’s why my colleagues upstairs at Nature are delighted with theirs, as at 36.101 it puts them ahead of their competitors. And also why journals with impact factors of 1 or 2 can also be delighted, as such an Impact Factor can be the highest in their specialist field. But when I looked into Impact Factors I realised that no protocols publications had them, so as a comparative factor, our impact factor would be rather meaningless. When and why would people cite protocols? We certainly wouldn’t be cited because we presented exciting new data and reported a major advance in scientific knowledge. I hoped that we’d be cited if we were used – and so lots of use would lead to a high impact factor. We’d obviously only get cited if the experiment worked, and so a higher impact factor would indicate good, reliable protocols. But, maybe we wouldn’t get cited, as citing the source of your protocol didn’t seem to be the ‘done thing’ at the time.

Five years down the line – with a 2 year ISI Impact Factor of 8.362 – I am pleased to report that we are being cited. We have an Immediacy Index of 1.547 and "Article Influence Score ":https://en.wikipedia.org/wiki/Article_influence_score of 3.799. We’re all rather pleased, as we know our Impact Factor matters to some of our authors. We’re also curious – which articles have people been citing? Are they the methods people have been using? Have they been getting good results? How do these articles correlate with the articles that are most downloaded from our site?

Our most highly cited article according to ISI is Locating proteins in the cell using TargetP, SignalP and related tools by Emanuelsson et al. This is different from the most highly cited article according to Scopus – "Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources ":https://dx.doi.org/10.1038/nprot.2008.211 – by Huang et al., although this comes in at second place. At third we have "Analyzing real-time PCR data by the comparative C-T method ":https://dx.doi.org/10.1038/nprot.2008.73 by Schmittgen et al. And in fourth, "In-gel digestion for mass spectrometric characterization of proteins and proteomes ":https://dx.doi.org/10.1038/nprot.2006.468 by Shevchenko _et al. _and in fifth, Protein structure prediction on the Web: a case study using the Phyre server. Interestingly there are lots of bioinformatics protocols in this list.

Of course, our 2010 2 year Impact Factor is only based on articles published in 2008 and 2009, so only two of those protocols, those in second and third place, will have been included. And also, only the cites in 2010. I was – and remain – curious about our five year impact factor. Whilst methods are developing all the time, protocols do tend to have a long life. A tweak here and there (hopefully highlighted as a comment to the protocol), but we hope our more useful protocols will still be in use, and cited, 5 years on. I guess we’ll have to wait and see

Happy Birthday Nature Protocols – 5 today!

Posted on by

Five years ago today we launched Nature Protocols. It was the culmination of 6 months of hard work for the editorial team. Dot Clyde and myself had both started work at NPG at the start of January with a long list of things to sort out. First we had to decide on our scope, answering questions such as do physicists use protocols? Next we had to decide on a format. Although we had a rough guide from our colleagues on Nature Methods who had published some protocols, we had much more freedom. No page budgets and an online format. Dot had just left the lab, as had the rest of the editorial team we recruited (Bronwen, Alex, Andy, Baldo and Hannah), so there was much discussion of what scientists REALLY wanted to know about. Next there was peer reviewing – what would we ask the peer reviewers to do? Ideally we’d have liked them to repeat the procedure, but clearly that was not practical (although on occasion we have had peer reviewers do just that!) And then there was presentation – sorting out exactly how we would present all that information we gathered. Plus who was going to copy edit it all. Then of course, last, but most certainly not least, who would we invite to write? And would they agree to write for this new type of publication?

Thankfully many of the authors we approached were as enthusiastic about the concept of Nature Protocols as we were. We quickly received plenty of manuscripts to send out to peer review (helped by the fact that the protocols were already written and much tried-and-tested in our authors labs). Many of the peer reviewers we approached were pleased to help, and on 12th May we put our first example protocols on our web page. It was great to finally have some actual content, alongside the information about what we were doing.

Then on June 30th 2006 we published our first batch of content – a whopping 87 Nature Protocols protocols. Over the following months we published many more so by 29th December we’d published 322. By January 2008 we had over 1000 protocols on our site, including almost 800 Nature Protocols protocols.

In addition, we’d also worked on our system for researchers to post protocols-in-progress onto the site. Then called the Protocols Network, this launched with 14 protocols, all contributed by authors of recent papers published in other Nature publications. Today that system is bigger and better than ever before, and was recently relaunched as the Protocol Exchange, including lots of added featured we didn’t quite get round to including in 2006.

So, 5 years on, where is Nature Protocols now? The editors are all a little older (here we all are back in 2006):

Copy (2) of S4200028.JPG

On the roof terrace in the summer of 2006. Back row, from left: Andy, Alex, Dot, Baldo. Front row, from left: Katharine, Hannah, Naomi (our first editorial assistant), Bronwen.

The feedback we’ve had from authors and users has been fantastic and largely positive, which is the best reward an editor can have. Our most cited article on Scopus – "Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources by Huang, Sherman and Lempicki ":https://dx.doi.org/10.1038/nprot.2008.211 – published in December 2008 has been cited 847 times. We’ve published old favourites, such as the single step method of RNA isolation by acid guanidinium thiocyanate-phenol-choroform by Piotr Chomczynski & Nicoletta Sacchi and the latest protocols published in Nature, such as "Kazutoshi Takahashi, Keisuke Okita, Masato Nakagawa & Shinya Yamanakas’ protocol for the induction of pluripotent stem cells from fibroblast cultures ":https://dx.doi.org/10.1038/nprot.2007.418. We’ve followed dramatic progress in certain fields – for example, the new protocols available using next generation sequencing have revolutionised research progress. It’s now possible to rapidly sequence single cells, as demonstrated by Fuchou Tang, Catalin Barbacioru, Ellen Nordman, Bin Li, Nanlan Xu, Vladimir I Bashkirov, Kaiqin Lao & M Azim Surani who used RNA-Seq analysis to capture the transcriptome landscape of a single cell. We continue to enjoy following the progress of stem cell research, which, as well as making dramatic progress actually generating stem cells, has produced improved protocols for researchers in other areas, such as the ability to generate gene knockout rats, described by Chang Tong, Guanyi Huang, Charles Ashton, Ping Li & Qi-Long Ying.

And the future? Well, I’ve taken a step back now and so that’s a question for our current Chief Editor, Chris Surridge, but I am sure we will continue to concentrate on our core aim back from 2006 of “publishing the protocols being used to answer outstanding biological and biomedical science research questions, including methods grounded in physics and chemistry that have a practical application to the study of biological problems”. All that’s left for me to do now is give a final thanks to all who have contributed to the success of Nature Protocols so far and say, heres to the next 5 years!

Focusing on stem cells

Posted on by

Since Nature Protocols launched we’ve been having a shall-we-shan’t-we discussion about the relative merits of article series. On one hand they seem such a great idea – bringing together lots of protocols in a specific area to make life easier for researchers in that area. On the other hand we worry they’ll alienate researchers working on other things – why do the mass spectrometrists get special treatment and not us? And how do you decide what to include on such a page – anything vaguely connected or just the hard core of a particular topic? Then finally, as an Editor I just find myself wondering why I am making more work for myself…

But back in 2009 we took the plunge and launched our Stem Cell Series page. At the time stem cells were all the rage – if you watched the TV news at night you’d think all diseases were on the verge of being cured by these fantastic new cells. Unless you lived in the US, where they were not viewed so kindly by some.

Recently the page has slipped a bit in priorities – launching the Protocol Exchange was quite a job. But we feel it’s now time to revitalise it. Stem cells remain a hot research topic, and the optimism about their potential remains. Each week more and more possible applications are being devised. I’m pleased to see that our recently published stem cell protocols are reflecting this increased variety. Currently featured, in addition to our typical protocols showing how to get your stem cells initially, or how to go from stem cells to your cells of choice, there are some intriguing ways to look at your stem cells. Some of these methods can be applied to cells other than stem cells so do check it out even if your favourite cell is not a stem cell.

The juries still out on whether series pages are a useful addition to our site. So, if you’ve not checked for a while, or if you’ve not looked at all, we do suggest you take a peek. And do let us know what you think – should we run series or not? And if so, what topics should we cover?

Hey look! Nature Protocols has a blog!

Posted on by

It is with some excitement and nervousness that I write the first test post on this new blog. We decided that it might be fun and useful for us Nature Protocols editors to have a place to write news about our content, and other stuff that might be relevant to Nature Protocols readers, and of course stuff that we as editors of Nature Protocols think is fun.

This first post, apart from being a test to see if the whole thing works, is also to ask a very important question:

What should we call our blog?

We have kicked the question around a little between us.

I am all for something straightforward like “Nature Protocols Blog”.

Dot on the other hand has suggested something more literary, perhaps “Method in the Madness” (quoting from Hamlet) though perhaps it will be more like “Madness in the Method”

Chris went all gothic and went for “The Laboratory Grimoire”.

Hannah suggested “Memoirs of a Protocolian” and is now regretting it. While Sam, who is certainly the most creative of our editorial team, hasn’t thrown her hat into the ring yet.

I am sure that you all have load of ideas, considerably better than ours! So tells us about them in the comment thread.

With many thanks and best wishes for all your experiments!

Bronwen