A recent report adds further evidence that assays relying on serial dilution and tip-based dispensing could be a source of irreproducibility, particularly in pharmacological assays.
A few days after I wrote the methagora entry below about our efforts to improve the reproducibility of published research, somebody pointed out a paper published last week in PLOS ONE that compared the results of automated serial dilution and plastic tip-based dispensing using a robotic sample processor to results obtained by an acoustics-based liquid dispenser. The latter is a technique using sound for noncontact liquid dispensing and is implemented in instruments such as those sold by Labcyte Inc., the employer of one of the authors on the manuscript. The dose-response data comparing the results of these two liquid handling methods, however, was previously published in patents by AstraZeneca on pyrimidine derivatives for inhibiting Eph receptors. The AstraZeneca results showed that data obtained on the 14 reported compounds via acoustic dispensing showed activities that were 1.5 to 276.5 times higher than data coming from serial dilution and tip-based dispensing.
What the PLOS ONE authors added to this story, besides promoting the research results to the press, was the computation of pharmacophores based solely on the two sets of activity data. The pharmacophore computed from the acoustic data was structurally similar to pharmacophores computed from x-ray crystallography data (for example, all these compounds contained hydrophobic binding domains) and was able to predict the activity of subsequent chemicals. In contrast, the pharmacophore computed from the serial dilution and tip-based dispensing data was very different, contained no hydrophobic domains, and was non-predictive.
What should one make of this? Well, it seems logical that hydrophobic domains could influence the results of serial dilution and dispensing through plastic tips via adsorptive or other effects. As one person commenting on the PLOS ONE paper states, such effects have been well documented and proper analytical technique calls for experiments to detect them.
This all reminds me of marketing for HP’s high performance dispenser that also forgoes serial dilution and instead uses inkjet printing technology to dispense undiluted reagents, presumably also via acoustics. HP promotes the increased reliability of this technique for generating dose-response curves but they don’t highlight the kind of effect documented by the authors of the PLOS ONE paper.
If these results are indicative of differences observed between these two types of liquid dispensing it seems that drug companies must be aware of them and are adapting their assays and protocols as necessary. But even if this is the case, there appears to be little evidence that academic researchers are worried about this.
In theory, one can certainly see the appeal of contactless dispensing but more hard data is needed to draw firm conclusions. This will require extensive side-by-side testing of different sample dispensing methods with many different compounds.
At a minimum, researchers need to be cognizant of this potential problem and report how they dispensed their reagents when reporting results from these kinds of pharmacological assays. Better yet, they should repeat key experiments on different days and with different equipment.
Update: I just found out that Derek Lowe has a nice post about this paper over at In the Pipeline
