Peer-to-peer

In an interesting exercise in how a journal can develop criteria for peer-review in a fast-moving, area of considerable intrinsic uncertaintly, the editors of Nature Reports Stem Cells posted an article last month that asked how one could declare human cells pluripotent, when the most robust tests are neither ethical nor feasible, calling on the US National Institutes of Health (NIH) to set standards. Now, in a post on The Niche blog, Defining pluripotency in human cells, the editors post some of the responses from researchers in the field. Here is one perspective, from Shinya Yamanaka of Kyoto University:

"This is an important, but difficult question. First of all, we don't know whether human ES cells are really ES cells or not. Because the lack of chimera experiments, we will not be able to answer this question. This means we lack a positive control. I have been telling my students that one of the worst experiments you can do is one without positive and negative controls."

The Niche also features an interview with Story Landis, head of the stem cell task force at NIH, about the world's response to the recent breakthrough result that human cells can be reprogrammed, including NIH's problem about how to develop criteria for funding in a research area that in itself does not support the US President's policy.

I will not be posting again on Peer to Peer until the New Year, so I wish you all a very happy Christmas and holiday season. If you feel the need for some interactive scientific discourse during this period of traditional peace for those involved in journal production, I can recommend Nature News (you can comment online on the articles) and Nature Network, where there are a range of groups, blogs and forums to suit any science-related interest, and to which you are welcome to contribute.

Posted by Maxine Clarke on December 19, 2007
Categories: Quality and value | Permalink | Comments (0)

From a post about peer review at the blog Unqualified Offerings:

"The simple fact of peer review is that it’s actually quite a modest hurdle. All you have to do is find one editor and a couple of reviewers who find the work plausible and well-executed. Once you’ve been on both sides of the process, you realize that it’s just a preliminary quality check, a first pass before it’s put out there for a wider audience. Some laymen seem to attribute too much significance to it, and other laymen seem to recoil against that misperception by concluding that peer review is too weak of a system. The truth is that it’s not supposed to be a stringent filter. It’s just supposed to be a first pass."

Posted by Maxine Clarke on December 18, 2007
Categories: Perspectives | Permalink | Comments (0)

Following on from my Peer to Peer post of a couple of weeks ago "How not to mix politics and science", I note (via Action Potential) that on 5 December, Daniel M. Cook et al. reported a highly relevant study in PLOS One 2(12): e1266. doi:10.1371/journal.pone.0001266 .
Cook et al. analysed 1,152 most-searched newspaper stories. They found that "funders of the research were identified in 38% of stories, financial ties of the researchers were reported in 11% of stories, and 5% reported financial ties of sources quoted. Of 73 stories not reporting on financial ties, 27% had financial ties publicly disclosed in scholarly journals." The authors note that many journals themselves do not require authors and reviewers to declare competing interests , and that journalists work under many different constraints, but firmly conclude that "news reports of scientific research were incomplete, potentially eroding public trust in science."
In his discussion of this work, Noah Gray of Action Potential writes: "Science journalism plays an enormous part in public opinion, which influences the actions of politicians (sometimes), who have some power to control major chunks of research funding. Therefore, public trust and respect are essential for the long-term growth and stability of scientific funding, especially from the government. So let's have the press play their part both in providing full disclosure and refraining from publishing scientifically-dubious (but headline-grabbing) stories, leaving the "spin" for the politicians."
The Nature journals' competing interests policy is here, at our authors' and peer-reviewers' website. We state:
"In the interests of transparency and to help readers to form their own judgements of potential bias, Nature journals require the authors of most articles to declare any competing financial interests in relation to the work described, by sending the author a form to complete and sign before publication of the article.
In cases where the authors declare a competing financial interest, a short statement to that effect is published as part of the printed article, with a more detailed version available online. If no such statement is present in the article, the authors have declared to the editors of the journal that they do not have any competing financial interests."
However, for peer-reviewers: "The Nature journals invite peer-reviewers to exclude themselves in cases where there is a significant conflict of interest, financial or otherwise. However, just as financial interests need not invalidate the conclusions of an article, nor do they automatically disqualify an individual from evaluating it. We ask peer-reviewers to inform the editors of any related interests, including financial interests as defined above, that might be perceived as relevant. Editors will consider these statements when weighing reviewers' recommendations."

Posted by Maxine Clarke on December 12, 2007
Categories: Ethics | Permalink | Comments (0)

In his Nature Network summary of a recent publishing meeting, Yiorgos Apidianakis describes his opinion that a peer-review "score" is a more desirable (and efficient) indicator of scientific excellence than the currently used impact factor of the journal that publishes the work. Charles G. Jennings's wrote, as part of Nature's peer-review debate, “It is common to bemoan the over-reliance on quantitative markers such as impact factors for assessing scientists’ abilities (and indeed there is much to bemoan), but until committee members have time to read every paper on every applicant’s CV, they will have to rely at least in part on proxy indicators.” Dr Apidianakis believes that an ideal indicator would be a score from a unified peer-reviewing system, or a central agency "that will thoroughly, rigorously and objectively evaluate any given work to be published, using again specialized scientists as reviewers. Having an evaluation score from such an agency, scientists can include this score to their publication record."

Dr Apidianakis thinks that such a system could work in practice, using the example of the NIH evaluation scoring system -- while admitting it would be very expensive. Irrespective of this practical obstacle (as well as others) I wonder how the quality of the agency's "score" can be standardized? The establishment of, in effect, one giant journal of research, with all the world's scientists signed up to it as peer-reviewers, is a stimulating concept. But, given the many divergences of views within fields, how could a centralized scoring system work? The IPCC has attempted a similar kind of approach for one discipline, climate change -- even though most scientists in the field broadly agree with the IPCC's assessment of research output, this consensus requires massive bureaucratic baggage, including many international meetings and vast reports justifying decisions. Yet there is a substantial minority of scientific dissentors, and many members of the public are sceptical of a unified approach resulting in "science by endorsement".

The current peer-review system works very well "in miniature", whether as operated by those journals able to call on the most thoughtful scientists in a field who do the most cutting edge research, or by literature reviews or research ranking services, usually written or operated by one or a few individuals. Scientific research itself has benefited from large author collaborations across the whole spectrum of disciplines, from astronomy and nuclear physics to genomics and cell signalling, facing the challenge of making sense of vast quantities of data. Are there the same intellectual and innovative advantages to be gained by a single-managed peer-review system for all of science?

Posted by Maxine Clarke on December 04, 2007
Categories: Quality and value, Systems | Permalink | Comments (0)