Where’s your ground truth?

Posted on 28 Oct 2011 by Daniel Evanko

When using or developing experimental and observational methods it is crucial to assess the method performance in an effort to ensure that the information it provides reflects reality. For experimental biologists this often means conducting carefully chosen control experiments with alternative methods or different experimental settings. More rigorous assessment, particularly for high-throughput or large-scale methods, often requires the use of ‘ground truth’ or ‘gold standard’ data sets. But talk to different people and you will get different answers regarding what ‘ground truth’ or ‘gold standard’ data is. This often includes a nice historical explanation of where the term ‘ground truth’ comes from.

For developers of signal processing and image analysis algorithms though, the situation is clearer; the ground truth is the signal or image you start with. But add a living system into the mix and things get far more complicated. The Editorial in the November issue of Nature Methods discusses the challenges facing developers and users of algorithms for automated analysis of biological data, with a focus on image data. In short, traditional ground truth data is often insufficient. The addition of integrated-editing and change-logging capabilities to these software tools can increase the quality of the analysis, aid further algorithm development and increase the likelihood of biologists adopting the software in the first place.

Academic inventions

Posted on 29 Sep 2011 by Daniel Evanko

Over the past 30 years the ties between academic research and commercial enterprise have increased enormously. Much of this increase has involved attempts by universities to capitalize on the intellectual property created by their research scientists using the US patent system. The Editorial in the October issue of Nature Methods discusses this change and the challenges facing academics interested in commercializing their innovations.

The America Invents Act was signed into law on September 16 by President Obama as the Nature Methods Editorial went to press. As discussed in the Editorial, this law introduces a fundamental change in US patent law that impacts how US academics and their technology transfer offices will handle their intellectual property once the law goes fully into effect a year from now.

Overall, the harmonization of US patent law with the rest of the world should greatly simplify patent claims. But it also presents challenges and fails to fix some aspects of patent law that make little sense, like forcing the same 20-year patent lifetime on classes of inventions that display huge disparities in the time and cost of moving from patent filing to commercial product and the corresponding difference in commercial lifetimes. The implementation of different patent lifetimes for different classes of inventions, for example pharmaceuticals versus computer technology and processes, would help correct severe imbalances in the current system. But given the years required to obtain passage of the America Invents Act, further significant changes are likely years away.

The links below have additional information and commentary on patent law and commercialization in academia.

US switch to first-to-file patents could cause minor shake-up Nat. Med. 17, 906 (4 Aug 2011)

New models emerge for commercializing university assets Nat. Biotechnol. 29, 774 (8 Sept 2011)

Patent reform on the brink Nat. Biotechnol. 29, 778 (8 Sept 2011)

The Effects of the America Invents Act on Technological Disclosure Patently-O Blog (8 Sept 2011)

Patent reform bill passes US Congress – September 09, 2011 Nature newsblog (9 Sept 2011)

Patents Directed to Human Organisms Patently-O Blog (9 Sept 2011)

Guest Post – To Promote Progress in Science and Job Creation Patently-O Blog (12 Sept 2011)

Patent medicine Nature 477, 249 (14 Sept 2011)

New Patent Law Could Change How Academics Commercialize Discoveries ScienceInsider (14 Sept 2011)

Patent Reform Shuffles Who Is First in Line Science 333, 1559 (16 Sept 2011)

Guest Post: Preclusive Inventor Disclosure Under Leahy-Smith Patently-O Blog (22 Sept 2011)

Efficiency through analysis

Posted on 28 Apr 2011 by Daniel Evanko

The May Editorial in Nature Methods discusses how the overall efficiency of research can be improved by comparative analysis of research method and tool performance.

Although such analysis studies aren’t considered as ‘sexy’ as basic exploratory research, the benefits for and gratitude from the community can be profound. Large well-funded laboratories are more likely to have the resources to perform such analyses and should not discount the advantages to performing such studies and publishing the results.

Nature Methods has published several such analysis studies in the past. A (probably incomplete) selection is listed below. We will strive to publish even more in the future. Our ‘Analysis’ article type is actually dedicated to these kinds of studies. We encourage communities and labs to both contribute such analyses and suggest methodological areas that would benefit from them. The selection below may provide some inspiration.

2005
Multiple-laboratory comparison of microarray platforms
doi:10.1038/nmeth756
Independence and reproducibility across microarry platforms
doi:10.1038/nmeth757
Comparative evaluation of mass spectrometry platforms used in large-scale proteomics investigations
doi:10.1038/nmeth785

2006
A guide to choosing fluorescent proteins
doi:10.1038/nmeth819

2007
Reproducible isolation of distinct, overlapping segments of the phosphoproteome
doi:10.1038/nmeth1005
Use of simulated data sets to evaluate the fidelity of metagenomic processing methods
doi:10.1038/nmeth1043

2008
Cyclic nucleotide analogs as probes of signaling pathways
doi:10.1038/nmeth0408-277

2009
Cost-effective strategies for completing the interactome
doi:10.1038/nmeth.1283
A HUPO test sample study reveals common problems in mass spectrometry-based proteomics
doi:10.1038/nmeth.1333

2010
Comprehensive comparative analysis of strand-specific RNA sequencing methods
doi:10.1038/nmeth.1491
Microbial community resemblance methods differ in their ability to detect biologically relevant patterns
doi:10.1038/nmeth.1499
Validation of two ribosomal RNA removal methods for microbial metatranscriptomics
doi:10.1038/nmeth.1507

2011
Chemically defined conditions for human iPSC derivation and culture
doi:10.1038/nmeth.1593
Two-photon absorption properties of fluorescent proteins
doi:10.1038/nmeth.1596

DNA origami on the rise

Posted on 25 Feb 2011 by Daniel Evanko

Nanotechnology is all the rage these days but its use by practicing biologists is still very limited. A recent entry in the nanotechnology arena is DNA origami, a method for creating nanostructures out of DNA that is more accessible than previous methods and allows larger and more complex structures to be created with greater ease.

In the April issue of Nature Methods you will find a primer to DNA origami that provides an excellent introduction to this technology with valuable practical advice on designing and synthesizing DNA nanostructures using the DNA origami methodology. We hope that this primer will stimulate biologists or others new to this field to take a look at this technology and dream up exciting new applications.

One of the crucial steps of DNA origami is isolating your properly folded structure. A Correspondence by William Shih, one of the pioneers of DNA origami, describes some simple but very useful modifications to an agarose gel electroelution method that many people use for isolating PCR products or small DNA fragments from restriction digests. These changes greatly increase the efficiency of isolating intact large DNA nanostructures compared to existing methods.

Finally, the Editorial discusses the prospects of DNA nanostructures created using DNA origami as biological research tools.

Based on the number of posters describing applications of DNA origami at the 2010 Gordon Research Conference on Single Molecule Approaches to Cell Biology, compared to previous years the biological community and the single molecule biophysics community in particular is showing interest in the methodology. Only time will tell if it fairs better among biologists than other promising nanotechnology tools and methods.

We’d like to know what our readers think of the biological research prospects of this technology, or other nanotechnology tools and methods for that matter. Tell us what you think.

Research collaboration

Posted on 02 Feb 2011 by Daniel Evanko

The Editorial in the February issue of Nature Methods discusses the critical role that interdisciplinary collaboration plays in modern biomedical research. Although there are certainly notable exceptions, researchers seem to be increasingly using collaborations with experts outside their own area of expertise to bring new insights and technologies to their research projects. Increasing numbers of studies claim to back up the assumption that collaboration is beneficial.

As highlighted in a 2008 News Feature in Nature, collaborations can also go horribly wrong and an accompanying editorial urged researchers to take some basic steps to avoid unforeseen complications.

At Nature Methods we were curious if the methods papers we had published showed any evidence of benefits arising from interdisciplinary collaboration. We calculated the number of citations per year that each of our research papers published between 2004 and May 2010 had received and analyzed the author patterns of those in the top and bottom quartiles. As briefly described in the February Editorial we detected what seemed to be a positive effect of interdisciplinary collaborations on the apparent impact of the work. This was by no means a rigorous study and there are many caveats, but it certainly suggests that even in methods development, collaboration is beneficial.

Do you have anything to share about your experiences with collaboration? We’d love to hear them.

Nobel thoughts

Posted on 28 Oct 2010 by Daniel Evanko

The Nobel Prize is quite possibly the most anticipated annual event in the scientific community. This year the winners again highlighted the importance of methodological development in scientific progress. Remarkably, the physics, chemistry and medicine prizes all rewarded method and tool developments. This continues, and possibly strengthens, a trend that has become more evident in recent years.

An editorial in the November issue of Nature Methods provides our thoughts on the Nobel Prize and suggests that the addition of a prize dedicated to biology might reduce some of the strain the prize has been experiencing recently and help protect the prize from an erosion of the community support it relies on.

What do you think? Is it ill advised to tamper with something of such stature and history or is it a long overdue change?

Methods and more

Posted on 30 Jul 2010 by Daniel Evanko

This month’s editorial describes recent changes that have been taking place inside the journal, most notably the addition of two new journalistic pieces “The Author File” and “Points of View”. The latter is a new monthly column with tips on how to graphically present scientific data written by Bang Wong. More information about Bang can be found on his website.

As Nature Methods strives to provide more useful and engaging content for our readers we hope you will let us know how we are doing, and what we could do better. We welcome your input and suggestions.

Resurgent rats

Posted on 27 May 2010 by Daniel Evanko

Although rats are detested, or at least tolerated, by the majority of people, some individuals find much to admire in them. Among these people are researchers who rely on the rat as an excellent animal model for biological research. The Editorial in the June issue of Nature Methods describes how genetic technologies are opening up new possibilities for research using rats and how researches could benefit by considering rats for their own study. Below is a limited selection of rat resources for those wishing to find out more about this indespensible laboratory animal.

Recent Articles

The knockout rat pack – 2010 News piece in Nature Medicine

The genome sequence of the spontaneously hypertensive rat: Analysis and functional significance – 2010 research article in Genome Research

National BioResource Project-Rat and Related Activities – 2009 Review article in Experimental Animals

Rats! – Editorial in Disease Models & Mechanisms

Return of the rat – 2009 story at Nature News

Selected Web Resources

Rat Genome Database – Repository of rat genetic and genomic data. Provides tools to search, mine, and analyze this data as well as information on genomic mapping strains and rat physiology.

Rat Resource and Research Center – Archiving and distribution of high quality, well characterized inbred, hybrid and mutant rats to investigators. [Note: A new and improved website is just weeks away.]

National BioResource Project for the Rat in Japan – Archiving and distribution of rat strains as well as phenotypic and genetic characterization of strains. All information provided in a publicly accessible database.

Rat Genome Project – Genome of the Brown Norway rat (Rattus norvegicus) produced by the Rat Genome Sequencing Consortium (RGSC).

Consortiums

EURATRANS: European large-scale functional genomics in the rat for translational research

The European Rat Tools for Functional Genomics (EURATools) consortium

Knock Out Rat Consortium (KORC)

Supporting young scientists

Posted on 21 Mar 2010 by Daniel Evanko

People have been expressing concern for years about the the continual increase in the age of US scientists receiving funding from the NIH. See for example this post and the links contained therein. Part of the concern is due to the observation, highlighted by a 1993 study, that most scientists do their groundbreaking work early in their careers and these people aren’t being adequately supported by the current system.

The National Academies organized a public workshop in 2004 to explore ways to address this issue and released the “Bridges to Independence” report in 2005 with a chilling but probably not too farfetched possible future outlined in the Foreward. The report called for many useful changes in funding and support by 2010 in an effort to promote effective independence for early career investigators. The NIH has implemented some of thse recommendations. Other countries are also attempting to improve funding prospects for young investigators.

An editorial in the March issue of Nature Methods discusses a different model, pioneered by EMBL and adopted to varying degrees by Janelia Farm and the National Institute of Biological Sciences in Beijing, which involves selective recruiting of talented scientists at an early stage of their careers and providing them with a high level of support that frees them from the need to compete for funding with established scientists and focus instead on high risk innovative research.

Neither model is sufficient on its own and even together they won’t fix the problem overnight but each has unique benefits and stronger efforts on both fronts would help turn things around before we find ourselves in a future similar to that envisioned in the “Bridges to Independence” report.

iPhones in the lab

Posted on 28 Jan 2010 by Daniel Evanko

Do you use your iPhone (or other smartphone or mobile computing device) in the lab? This month’s editorial notes how large numbers of scientists seem to have an iPhone or other mobile device capable of running quite sophisticated applications, or apps. Increasing numbers of these apps are targeted at biologists and some are even intended for use at the lab bench; and lists of recommended apps are popping up on blogs and other sites. Check out the links below for a sample.

22 iPhone Apps for Science Geeks – July 11, 2008

More iPhone apps for scientists – October 13, 2008

5 Bio-Related Apps for your iPhone/iPod Touch – November 4, 2008

iPhone and research – July 24, 2009

iPhone apps every biologist needs – October 9, 2009

10 Best iPhone Apps for Science Majors – December 23, 2009

Some recently released apps that don’t appear in the lists above are:

Bio-Rad PCR – Practical guidance for performing PCR and qPCR

NEB Tools – Double digest finder and restrictions enzyme finder tools

ChemMobi – Search for chemical information by name or ID. View selected properties, MSDS information and structure.

LabCal & LabCalPro – Various laboratory calculation functions molarity, moles, stock dilutions, pH & g-force

GeneticCode & GeneticCodePro – Reference tool the nucleic acid codon table and amino acid properties

But how likely is it that bench researchers will actually use an expensive personal mobile computing device like an iPhone in the lab environment? Since we are no longer in the lab ourselves, we wonder what the current generation of grad students and post-docs are doing. Is your iPhone useful in the lab? What about a similar portable device? What apps do you use?

Although there has been a lot of noise surrounding the new Android-based phone from Google we were unable to find an apps intended for use in the lab on that platform, with the exception of seemingly hundreds of scientific calculator apps. We would love to hear from any readers who are familiar with scientific apps available for platforms other than the iPhone.

Yesterday Apple announced the long anticipated iPad mobile computing device. This tablet computer can run iPhone apps in addition to providing a far larger screen and the capability to run more powerful applications than the iPhone. It seems unlikely that such a device would become as ubiquitous among scientists as the iPhone since it doesn’t double as a phone. However, it has definite advantages as a dedicated laboratory tool and is more suitable for reading journal articles than the iPhone.

Speaking of reading journal articles, the nature.com app should be available for download from the Apple app store on February 1. We’ll keep you posted.

nature.com blogs

Author Archives: Daniel Evanko