How to deal with technical criticisms of published work
"Scientific publishing depends on expert peer reviewers. Instead of perpetually arguing about the reliability and fairness of peer review, authors, editors and referees should seek to optimize this time-tested system." So opens the January editorial of Nature Neuroscience (12, 1; 2009).
The editorial discusses the media reporting of a ferocious argument about the merits of a paper published in Cell , and a subsequent blog debate hosted by The Scientist. The controversy between scientists in this discipline concerning this paper "has again ignited a debate on the flaws of editor-managed anonymous peer review", write the Nature Neuroscience editors. "We maintain, however, that despite occasional unfortunate lapses, anonymous peer review remains the best quality-control process that we have." The editorial goes on to discuss how journals can best optimize the process.
In Nature News this week (457, 245; 15 January 2009) another technical dispute is discussed, this time concerning a widely circulating preprint attacking much of the published research in social neuroscience involving functional magnetic resonance imaging (fMRI). This affair is exacerbated by the fact that the circulating preprint is not due to be published, with responses from the authors of some of the criticized studies, for another nine months, according to the Nature News story. The criticisms have already been covered in publications such as Newsweek, as well as the inevitable blogosphere outpourings - and at least some of the criticized authors say the first they heard of the preprint was when they were contacted by journalists.
How scientific reports should be peer-reviewed and, after publication, scrutinized are matters that are decided upon by the journals, their editors, and their publishers and/or societies - in the case of articles submitted to and published in the Nature journals, the peer-review process is described here, and the post-publication corrections process here. Good journals have processes for investigating technical criticisms and complaints about the papers they publish. Nature's, for example, is here. Often, a resolution is not clear-cut at the outset, when the complaint is first received by the journal, however clear it may be in the mind of the complainer. A proper outcome depends on independent peer-reviewers, as well as editors, examining the complaint together with a measured response from the study's authors. Playing out such investigations in the kangaroo courts of the popular press, or in unfettered comments on the Internet between people who have been described as "recreationally outraged", not only obscures logical, technically informed investigation, but unnecessarily exacerbates emotions and arguments so that, in the end, all that is remembered is the heat - not any light.
Comments
Word of the day:
exacerbates
------
Posted by: Andrew Sun | January 19, 2009 02:20 PM
The Scientist really have no clue if they're hiding their blogs behind a registration wall...
Posted by: rpgrant@gmail.com | February 9, 2009 08:08 AM