A peer-reviewed study that claimed “American scientists are significantly more prone to engage in data fabrication or falsification than scientists from other countries” caused surprise when it dropped into Nature’s inbox. Could a researcher in the United States really be more likely to publish fake research?
In fact, no. The study (G Steen, J. Med. Ethics, 2010; doi: 10.1136/jme.2010.038125) looks at retractions of research papers in the database PubMed over the last decade and finds that retractions by US authors have a high fraud-to-error ratio (a third of US retractions were due to fraud rather than some sort of mistake).
But this does not mean that any US scientist is more likely to engage in data fraud than a researcher from another country. Indeed, a check on PubMed publications versus retractions for frauds suggests that s/he may be less likely to do so (though the statistical significance of this finding has not yet been tested).**Bob O’Hara, blogging at ’Deep Thoughts and Silliness", has run the analysis.
How did the statement about ‘more prone to engage in data falsification’ make the paper?
The study’s author is Grant Steen, a science writer and president of Medical Communications Consultants, a company that provides medical writing services. It turns out that he interprets the phrase ‘more prone to…’ as simply the statement that US scientists as a group produce the most frauds and retractions. As to the question of which scientists are most likely to produce frauds, his data do not address that point, he agrees.
In January this year, Steen looked at all retractions from papers published between 2000 and 2010 in the English language, indexed in the PubMed database by first author affiliation. He also identified whether the 788 retractions he found at that time were due to an error (including text plagiarism) or fraud (data plagiarism, data fabrication, or data falsification).
Here are Steen’s figures:
It’s clear that the United States produces most retractions and most frauds. And as Steen points out, the figures show that one in three retractions from an author affiliated in the United States were attributed to fraud instead of error. For authors affiliated in other countries, the error:fraud ratio is higher. (Though India’s ratio looks worse than the United States, that difference is not statistically significant, Steen says).
One point to make is that, within the subset of retractions, the fact that the US fraud rate is relatively high may only indicate that sloppy errors are relatively low, so that retractions tend slightly more towards fraud cases. But at least, Steen says, it’s clear that having a ‘strong’ research infrastructure does not protect against fraud.
However, it’s worth also pointing out that this doesn’t mean that US scientists are more prone to fraud than those from other countries. PubMed is dominated by papers published by US-affiliated researchers, so that their absolute fraud rate may be quite low compared to other countries. Steen does not address this point, but I quickly checked out the figures, using this PubMed tracker website. Here they are for the top seven countries including my (non-peer-reviewed) figures for their total number of publications in English:
It appears that US researchers have a lower fraud and retraction rate than authors affiliated with China, India, and South Korea (update: for a statistical analysis see Bob O’Hara’s blog).
Apart from the (to my ears) misleading statement about US researchers’ prone-ness to fraud, Steen’s paper does contain some other intriguing findings. For instance, roughly 53% of fraudulent papers were written by a ‘repeat offender’ who had written other retracted papers. Retracted papers were more likely to appear in journals with a high impact factor – though this might be because lower impact-factor journals tend to retract fewer papers.
“The overall rate of retraction is really very low and that’s a message that should not be lost in this,” Steen adds.
Update: I’m reminded that the pattern by which retracted papers are more likely to appear in journals with a high impact factor has been noted before (M. Cokol et al. EMBO Rep. 5, 422–423; 2007) – see Nature 447, 236-237 (17 May 2007).