{"id":28,"date":"2011-04-22T00:07:37","date_gmt":"2011-04-22T00:07:37","guid":{"rendered":"https:\/\/blogs.nature.com\/tradesecrets\/2011\/04\/22\/academic_bias_and_biotech_failures"},"modified":"2012-01-12T16:29:27","modified_gmt":"2012-01-12T16:29:27","slug":"academic-bias-and-biotech-failures","status":"publish","type":"post","link":"https:\/\/blogs.nature.com\/tradesecrets\/2011\/04\/22\/academic-bias-and-biotech-failures","title":{"rendered":"Academic bias and biotech failures"},"content":{"rendered":"

I just met with an entrepreneur who was the founding CEO<\/span> <\/span>of a company created around an academic lab\u2019s discoveries. It was fascinating new approach to drugging hot receptor targets. To protect the innocent I won\u2019t mention the names, but Atlas Venture looked at in back in 2008 and, although intriguing, we ended up passing on the deal. Thankfully, because we missed a bullet \u2013 it recently was shut down.<\/p>\n

The reason: the foundational academic science was not reproducible outside the founder\u2019s lab.<\/p>\n

The company spent $5M or so trying to validate a platform that didn\u2019t exist. When they tried to directly repeat the academic founder\u2019s data, it never worked. Upon re-examination of the lab notebooks, it was clear the founder\u2019s lab had at the very least massaged the data and shaped it to fit their hypothesis. Essentially, they systematically ignored every piece of negative data.<\/p>\n

Sadly this \u201cfailure to repeat<\/strong>\u201d happens more often than we\u2019d like to believe. It has happened to us at Atlas several times in the past decade.<\/p>\n

The unspoken rule is that at least 50% of the studies published even in top tier academic journals \u2013 Science<\/em>, Nature<\/em>, Cell<\/em>, PNAS<\/span><\/em>, <\/span>etc\u2026 \u2013 can\u2019t be repeated with the same conclusions by an industrial lab. In particular, key animal models often don\u2019t reproduce. This 50% failure rate isn\u2019t a data free assertion: it\u2019s backed up by dozens of experienced R&D <\/span>professionals who\u2019ve participated in the (re)testing of academic findings. This is a huge problem for translational research and one that won\u2019t go away until we address it head on.<\/p>\n

Reality is we live in a tournament model world of academic research: winners get the spoils, losers get nothing. Publish or peril.\u00a0\u00a0Grants are really competitive, and careers are on the line. Only positive findings are typically published, not negative ones.\u00a0 This pressure creates a huge conflict of interest for academics, and a strong bias to write papers that support the hypotheses included in grant applications and prior publications.\u00a0 To think there is only objectivity in academic research, and pervasive bias in industry research, is complete nonsense.<\/p>\n

But what about academic bias?\u00a0 Or the lack of repeatability of academic findings? I couldn\u2019t find a single paper in PubMed over the past few years.<\/span><\/strong><\/p>\n

So what can drive\u00a0the failure to independently validate the\u00a0majority of peer-reviewed<\/span><\/strong><\/p>\n

published academic findings?I\u2019m sure there are cases where it\u2019s truly fabrication or falsification of data, but as an optimist I believe that must be a tiny percentage: most of the time I think its just the influence of bias. \u00a0A few\u00a0possible hypotheses exist for how this bias could manifest itself:<\/p>\n

1<\/strong>. Academic investigator\u2019s directly or indirectly pressured their labs to publish sensational \u201cbest of all experimental\u201d results rather than the average or typical study;
\n2<\/strong>. The \u201cspecial sauce\u201d of the author\u2019s lab \u2013 how the experiment was done, what serum was used, what specific cells were played with, etc.. \u2013 led to a local optimum of activity in the paper that can\u2019t be replicated elsewhere and isn\u2019t broadly applicable; or,<\/p>\n

3<\/strong>. Systemically ignoring contradictory data in order to support the lab\u2019s hypothesis, often leading to discounting conflicting findings as technical or reagent failures.<\/p>\n

Importantly, how are venture capitalists who invest in biotech supposed to engage on cool new data when the repeatability is so low? Frankly, most VCs don\u2019t do early stage investing these days, and this resistance to fund early academic spin-outs is in part due to the insidious impact of the sector\u2019s high failure rate with academic reproducibility (a.k.a. \u2018bias\u2019). But for those of us who remain committed to early stage investing, I\u2019d suggest there are at least two key takeaways for VCs:<\/p>\n