Nature Methods | Methagora

Can doing things differently enhance reproducibility?

Intuitively most of us would agree that standardized conditions in scientific experiments ensure reproducibility. If I know how something was done and exactly replicate the setup and protocol, I should get the same results. The authors of a Perspective in the April issue of Nature Methods argue otherwise when it comes to behavioral experiments done with live animals.

Hanno Wϋrbel and his co-workers present their view that standardizing environmental conditions in animal experiments actually leads to poor reproducibility, since it makes the results more vulnerable to the most subtle local differences that can result in conflicting results between different labs. Instead, the authors advocate systematic environmental heterogenization. In their opinion this will increase reproducibility and thus decrease the ethical cost of animal experiments.

To prove their point the authors performed a data-mining experiment in which they group together the results from several behavioral tests performed in a multi-laboratory study with mice. They re-analyzed the published data by either maximizing or minimizing the environmental variation between the groups. The results from the heterogeneous group—those within the maximized environmental variation group—were less variable than those from the standardized group.

With this analysis Wϋrbel and co-workers challenge the current paradigm that standardization is essential in animal experiments. Their view is likely to spark discussion and we invite you to contribute to this discussion here.

Comments

There are currently no comments.