Scientists are irked over a paper claiming, as The New York Times reported on Monday, that “DNA’s power to predict illness is limited.” “Yes,” geneticists have replied. “What else is new?”
Geneticists don’t dispute the idea that genes aren’t the only factor that determines whether we get sick; many of them agree with that point. The problem, geneticists say, is not that the study, published on 2 April in Science Translational Medicine, arrived at a false conclusion, but that it arrived at an old, familiar one via questionable methods and is now being portrayed by the media as a new discovery that undermines the value of genetics. Here are the main criticisms of the new study and the resulting press coverage:
1. This study critiques the power of genomic medicine but does not contain any genome data. The paper is titled, “The predictive power of personal genome sequencing,” but it doesn’t include any sequence data. Instead, the authors analysed data on how often twins developed the same diseases. Because twins have very similar genomes but don’t always develop similar ailments, the authors, led by Bert Vogelstein and Victor E. Velculescu of the Johns Hopkins Kimmel Cancer Center in Baltimore, Maryland, assumed that the frequency with which the twins got the same illnesses reflects the power of their underlying genome sequences to determine their health. This assumption is not true (see point 4), and isn’t a good basis on which to dismiss the value of genome sequencing in the absence of data from large genome-sequencing studies, which are just now getting underway.
“Let’s fast-forward a year or two, when we’ve sequenced a million or two million people in whole-genome sequencing studies,” says Eric Topol, a cardiologist at Scripps Health in La Jolla, California, and author of The Creative Destruction of Medicine: How The Digital Revolution Will Create Better Health Care. “Then let’s see whether or not the predictive capacity is limited, or limited for certain conditions but not others.”
2. This study is beating a dead horse. Many other studies have already found that genes alone don’t predict a person’s risk for developing most diseases very well. They’ve also specifically questioned the value of commercial genetic tests that promise to reveal users’ risk for various illnesses. The new study doesn’t acknowledge any of the previous studies that have already arrived at the same answer and have done a better job of it, geneticists say (see point 3).
3. The mathematical model used in the study is unrealistic. Geneticists have developed a slew of mathematical models that try to predict how likely a person is to develop various diseases. Scientists debate how well these models work, but the models are largely based on how diseases actually behave in the real world. The Vogelstein–Velculescu model is not, say statisticians.
Vogelstein, Velculescu and their colleagues first developed a model that poses a theoretical idea of how diseases might behave. They then tested their model against data from twin studies. The model divides the universe of human genomes into 20 groups, or “genometypes.” Each of the genometypes encodes a certain disease risk and occurs with a certain frequency, but the authors don’t know how often different genometypes carrying various disease risks occur. To figure this out, they ask which combinations of disease risk and genometype frequency are realistic by comparing them to what they actually see in twin studies.
The problem with this approach, statistical geneticists say, is that it uses flawed data to test unrealistic assumptions. Geneticists know how often certain genetic risk variants for various diseases occur in the general population, and how much risk each of these variants confers. The new model ignores this information, and instead allows diseases to behave in ways that differ from how they behave in real life. “The particular parameters in the model don’t really correspond to anything in terms of real world behaviour of genetic risk variants,” explains Luke Jostins, a statistical geneticist at Cambridge University, UK. “This divorces the model from population-genetic plausibility, making the results potentially meaningless.”
By ignoring information about how diseases act in the real world, the new model also allows the authors to sidestep some controversial unanswered questions, such as whether standard models overestimate the genetic contribution to disease in twin studies. That could be a nice feature of the model, geneticists say. But because of the limitations of twin data, combined with the authors’ flawed analysis of these data (see point 4), there’s nothing in the paper to ground the new model in reality. If this were the first-ever paper to try to define the limits of genetic-disease prediction, it wouldn’t be convincing, says Jostins, who also blogs at Genomes Unzipped. “It’s very hard to interpret this model,” he says.
4. The study doesn’t correct for errors that can affect twin studies. The study assumes that genetics is the sole factor that determines whether two twins develop the same disease. But twins also grow up in a common environment, and the study doesn’t account for this, as the authors admit.
It’s also rare for both members of a twin pair to develop the same disease. So even a study such as this, which combines data from many different twin studies, suffers from a relatively small overall sample size of affected twins. That lowers the statistical reliability of its findings and introduces unpredictable errors into the study, Jostins says. Again, there are ways to account for for these errors, but this study doesn’t try to do that.
5. The media coverage of the study could weaken support for genetic research. Geneticists have lobbed some pretty heavy artillery at the Science Translational Medicine study, even though it claims to affirm what they already know. That’s because the new study has received more press coverage than your run-of-the-mill statistical genetics paper, and geneticists are concerned that the coverage has overblown the study’s conclusions in ways that could harm public support for science. “I don’t see the harm in telling the public yet again that there is no such thing as genetic determinism,” says Leonid Kruglyak a geneticist at Princeton University in New Jersey. “But I worry about the message being distorted to mean that genes have no value, or that genetic research is not worthwhile.”
Follow Erika on Twitter at @Erika_Check.
