Can doing things differently enhance reproducibility?

Intuitively most of us would agree that standardized conditions in scientific experiments ensure reproducibility. If I know how something was done and exactly replicate the setup and protocol, I should get the same results. The authors of a Perspective in the April issue of Nature Methods argue otherwise when it comes to behavioral experiments done with live animals.

Hanno Wϋrbel and his co-workers present their view that standardizing environmental conditions in animal experiments actually leads to poor reproducibility, since it makes the results more vulnerable to the most subtle local differences that can result in conflicting results between different labs. Instead, the authors advocate systematic environmental heterogenization. In their opinion this will increase reproducibility and thus decrease the ethical cost of animal experiments.

To prove their point the authors performed a data-mining experiment in which they group together the results from several behavioral tests performed in a multi-laboratory study with mice. They re-analyzed the published data by either maximizing or minimizing the environmental variation between the groups. The results from the heterogeneous group—those within the maximized environmental variation group—were less variable than those from the standardized group.

With this analysis Wϋrbel and co-workers challenge the current paradigm that standardization is essential in animal experiments. Their view is likely to spark discussion and we invite you to contribute to this discussion here.

Defining the scientific method

The editorial in the April issue of Nature Methods explores the role of methodological developments in the evolution of the scientific method. Some have argued that the ability to collect massive amounts of data and combine this with powerful correlative analyses will make hypothesis-driven research in biology obsolete. Others say that such ‘investigation’ is no longer science.

We suggest that some middle-ground will probably win out and both forms of inquiry will prove useful in answering important biological questions. What is your opinion?

Top downloads for February ’09

It is always interesting to see how papers we publish are received by the community and one useful metric is the number of times a paper is viewed or downloaded. Authors also like to see this information when their paper was one of the most heavily viewed papers.

Each month we will report the original research papers we published that received the most unique views (HTML) and downloads (PDF) the previous month. This will be broken down between papers published in that month’s issue and papers published in prior months. Now for the most popular downloads for the month of February.

Top 4 research papers published in the February issue

1. Microfluidic control of cell pairing and fusion

2. Photoactivatable mCherry for high-resolution two-color fluorescence microscopy

3. A bright and photostable photoconvertible fluorescent protein

4. miRNA in situ hybridization in formaldehyde and EDC–fixed tissues

Top 10 research papers published prior to the February issue

1. Mapping and quantifying mammalian transcriptomes by RNA-Seq

2. Stable knockdown of microRNA in vivo by lentiviral vectors

3. Genome-wide analysis of transcription factor binding sites based on ChIP-Seq data

4. Stem cell transcriptome profiling via massive-scale mRNA sequencing

5. mirWIP: microRNA target prediction based on microRNA-containing ribonucleoprotein-enriched transcripts

6. Genome-wide profiles of STAT1 DNA association using chromatin immunoprecipitation and massively parallel sequencing

7. Whole-cell 3D STORM reveals interactions between cellular structures with nanometer-scale resolution

8. Bright far-red fluorescent protein for whole-body imaging

9. High-resolution mapping of copy-number alterations with massively parallel sequencing

10. A versatile tool for conditional gene expression and knockdown

Pittcon: Mixing the old and the new

After attending the Biophysical Society Meeting in Boston last week, I am now in Chicago at the annual Pittcon meeting. For those of you do not know about Pittcon, the focus of the conference is on technology development for the fields of biology, analytical chemistry, and nanotechnology. Being so broad it should come as no surprise to learn attendance is usually quite large – around 17,000 attendees are here now. This year sessions have included advances in mass spectrometry, in vivo imaging, nanotechnology sensors and surfaces, vibrational spectroscopy, biofuels, nanomedicine, magnetic resonance and biomarker discovery/analysis along with an exhibit hall requiring a tram to move people between sections (more below).

Continue reading

The social lives of fruit flies

It has been appreciated for a while that fruit flies interact with each other, but studying these interactions is quite difficult. In fact, this is a general problem in understanding animal behavior – the measurements are incredibly tedious to make and to quantify. In a paper published yesterday in Nature Methods (Dankert et al, 2009), Heiko Dankert, Pietro Perona, David Anderson and colleagues now automate the process, using the power of machine vision to analyse social interactions in pairs of fruit flies.

Put two wild-type male fruit flies together, and they will lunge, tussle and threaten each other, actions classified as aggressive (however, not all interactions between male flies are aggressive). Put a male and a female fruit fly together though and, predictably, their interactions will be largely of the courtship variety; the male circles the female, extends his wings in a courtship ‘song’, and sometimes, things work out!

Dankert et al show in their paper that they can take simple videos of fly-pairs, and use new software to accurately identify these stereotypical aggression and courtship actions. This allows them to observe many such fly-pairs and to describe these behaviors quantitatively. What’s more, the software detects expected differences in fly behavior. Male flies in which octopaminergic neurons have been silenced or where the fruitless gene is spliced into a female-specific form are much less aggressive. Male flies in which cholinergic neurons have been feminized now court other males. The software sees these differences.

Importantly, the measurements can be made quickly and without researcher bias or fatigue. For instance, Dankert et al estimate that experiments that would take them about a hundred hours to perform manually can now be completed in a few minutes. This opens up the study of these behaviors to genetic and other screens, and it will be interesting indeed to see how such studies play out.

Biophysical Society Meeting: A patch of déjà vu

Sunday afternoon at the 2009 Biophysical Society meeting in Boston, Massachusetts, I found myself wandering the aisles of the exhibit hall with an odd sense of “high-throughput patch clamping” déjà vu. Every row I walked down had at least one company demonstrating the latest in automated patch clamping instrumentation. Apparently, these automated approaches for patch clamping cells are becoming more and more common since their initial introduction in 2001 and here in Boston six different companies are showcasing their approaches. The systems range from being “adaptable” – capable of patching a range of cell types — to “high-throughput”— capable of patching up to 48 individual cells in 48 wells in a single run in their current versions but with a 96-patch system in the works.

For me this information was simply amazing. You see I have always thought there is a limit to what can be automated in the lab. Oh sure, liquid handling applications and reaction set up have benefited incredibly from robotic automation and many parts of microscopy are now automated as well. But patch-clamping – where a very narrow tube must be carefully placed on a single cell to measure the conductance of ions through a channel in response to a stimulus — is different. I can still recall one of my earlier research endeavors where our group was thinking about trying a single patch-clamp experiment only to find the technique required a tremendous amount of skill, experience and patience (Lacking all this, I was extremely thankful for the expert help of a patch-clamp specialist in the department). So you can understand my amazement at seeing these robotic systems capable of doing it all now. And by all, I mean the whole process in a single box. These platforms obtain cells from either a culture flask or a holding tray (cells can be cultured for hours in the instruments prior to the experiment starting), centrifuge the cells to remove culture media prior to patching, place individual cells into wells for testing, form a seal to the cell surface (in most cases this is accomplished using a specific amount of suction which differs from cell to cell and has to be calculated for each patching experiment), add any compounds to be tested to the wells and finally record the conductance across the membrane in each well. All this in a basically unattended format.

Although these systems are expensive, it turns out that they are not much more than two or three complete traditional patch-clamping apparatuses without the cost of a couple electrophysiologists. All this got me to wondering as I rode the escalator out of the hall at the end of the day — is there a limit to what can be automated?