Action Potential

Extraordinary claims require extraordinary evidence

A tangent related to the primate cloning paper has understandably received less attention, but deserves its own thread. In the same issue of Nature containing that paper, an accompanying editorial described how Nature, for the first time, implemented a relatively new policy by seeking the independent confirmation of this particular “high-risk” finding (or “strong claim”) during the review process.


Obviously, successfully cloning a primate is a strong claim. This policy is mainly in response to the travesty that occurred over fraudulent claims by South Korean researcher Woo Suk Hwang that he and his team had cloned human embryonic stem cells (published in Science in 2004 and 2005). That prompted Science to adopt a risk assessment template to target those findings that could be false, and thus necessitate further inquiry prior to publication. The Nature policy is similar. Therefore, when these top journals decide that a certain study is bound to have an astronomical impact, extra care and effort to scrutinize the original data, even independently replicate the findings, are required to avoid potential damage to the scientific community.

The Hwang affair no doubt damaged the community and was a complete embarrassment for Science. So this idea seems like a win-win situation for all involved – the journal, the authors, and the public. But I am a bit troubled over the idea of weighting certain discoveries and fields as being more important than others. I understand that stem cell cloning technology trumps rodent whisker barrel physiology in the eyes of the public, the politicians, and (arguably) the entire research community, but it seems like these types of policies have the potential to start as a trickle and eventually become more of an alarming flow. Once this procedure begins to demonstrate its merits and ease of implementation, what’s to stop the editorial board from casting a wider net with the strategy and start loosening the definition of “extraordinary findings”? I guess I am just uncomfortable with the bizarre scenario of journals soliciting experiments and orchestrating scientific research between groups, just to make sure that they are not embarrassed.

Don’t get me wrong, this policy will save a lot of people a lot of grief, if wielded with a strong and disciplined hand. I just don’t want it to become a regular part of the review process.

Comments

  1. Eric Thomson said:

    I would worry that authors would submit elsewhere as they wouldn’t want to risk taking years for someone else to replicate their findings before getting published! By then some graduate students on the paper would likely be well into their first post-doc, and their post-doc search could be less optimal because their big publication is still ‘under review.’

    On the other hand, it seems that the editors are well aware of this potential problem, stating “the corroborating experiment was straightforward and an expert was willing to do it on a timescale that would not delay publication of the paper.”

    As long as that remains the case, I guess this will serve only to expedite scientific progress by quickly injecting a replication step before the paper is even published.

  2. DSK Samways said:

    I disagree with this course of action for reasons highlighted by the commenter above; it simply adds another layer of bureaucracy to publication and slows down the process of disseminating new and exciting discoveries. Besides, is fraud so sufficiently widespread as to warrant such a clamp down?

    As for the Hwang affair, surely that was an embarrassment to nobody but Hwang, his co-workers and his institution? The process of science is inherently self-policing, and if the Hwang affair demonstrated anything, it was just how efficient that self-policing actually is. A startling discovery was made that, based on the data submitted for review, was ostensibly legitimate. It went to publication in a journal that was appropriate for the importance of the supposed discovery. Precisely because the discovery was so outstanding, other labs quickly set about repeating the data, and very quickly questions started to be raised. The questions were not given satisfactory answers, fraud was investigated and confirmed, and the paper was retracted. I would call this an example of the current system working quite well. After all, everybody knows that peer review doesn’t end with an acceptance for submission. That’s when the serious peer review begins in earnest, especially for ground-breaking claims.

    Lastly, I’m not sure journals should be responsible for pursuing ethical investigations; that’s the role of the institution and funding organisation.

    Report this comment Cancel report
    Your details

    Please confirm the words below

    In order to reduce spamming, this process ensures you are a real person and not an automated program.

  3. Noah Gray said:

    Please check out a continuation of this discussion over at the blog of DSK Samways, who commented above.

    I agree with a lot of the points you make. Frankly, this whole idea does make me nervous.

    As for journals, take my word, they definitely take it personally when a paper has to be retracted.

  4. DSK Samways said:

    “As for journals, take my word, they definitely take it personally when a paper has to be retracted.”

    I don’t doubt that. Regardless of where the fault lies, it’s a disappointment for all concerned. I also don’t doubt that, in the spirit of competition, there’s a certain sense of schadenfreude when a rival gets caught out with a bum publication. 😉

  5. Action Potential said:

    Pet sematary

    A woman in the US has decided that she loved her pit bull (named Booger) so much, that having him all over again is definitely worth the $150,000 price tag. I saw this story in the BBC, reporting how a…