Peer-to-peer

An article in the Financial Times, Science stifled? Why peer review is under pressure (11 June 2008), reports various recent criticisms of the peer-review system, including a letter to the newspaper by 25 distinguished scientists calling for a "global fund to support inspired scientists, free of peer review"; news of a Royal Society pilot scheme for a “blue skies” research fund, to avoid the "constraints of conventional peer review by using a generalist panel to consider proposals from any field, on the basis of their novelty and potential to open up new areas of science and technology"; and in the announcement of this year's Grand Challenges programme of the Bill and Melinda Gates Foundation, Tachi Yamada, the foundation's head of global health, is cited as saying "We've got to get around peer review – it’s anathema to innovation. Innovation has no peers, by definition.”
The Financial Times article goes on to identify various innovations in the peer-review process itself, being tried or in normal use by various publications. Scientists themselves, however, choose to publish in the highest quality journals rather than on the basis of their peer-review systems. Linda Miller, US Executive Editor of Nature, is quoted in the article:

Linda Miller, executive editor of Nature, agrees that scientists continue to seek publication in prestigious journals to enhance their own standing. They also concentrate on reading the best-regarded ones, precisely because their time is precious. “You want to be directed, to use the best journals as a filtering device,” she says. “I have been an editor for more than 20 years and I have handled a lot of papers. Every single one has been improved by peer review.”

Posted by Maxine Clarke on June 25, 2008
Categories: Perspectives | Permalink | Comments (2)

Ai Lin Chun, one of the Nature Nanotechnology editors, writes on Nature Network that the journal has received a few voluntary comments from referees recently on the topic of innovative (creative) papers versus those bridging the gap between 'pure' nanotechnology and applied clinical research. She explains that papers submitted to Nature Nanotechnlogy in the field of nanomedicine often struggle during the peer-review process for these and other reasons, providing a challenge for the editors in making their decisions about publication.
The main criteria for publication in Nature journals (Nature Nanotechnology included) are: originality; technical soundness; a substantial advance over previous work; conceptual novelty or unexpectedness; and broad interest or practical applicability.
One of the criticisms raised by Nature Nanotechnology's peer-reviewers has been that publishing innovative 'nanomedicine' papers may be exciting to physical scientists in the nanotechnology field, but would not be credibile within the medical and clinical community. This is an example of a common challenge that faces editors of interdisciplinary journals when setting publication criteria: an exciting advance in one field may be less than compelling to those in another. In the case of nanomedicine, there are some interesting perspectives expressed by researchers in response to Ai Lin Chun's post.

Posted by Maxine Clarke on February 15, 2008
Categories: Perspectives | Permalink | Comments (0)

FemaleScienceProfessor: Extreme Reviewing

In the post linked above, Female Science Professor writes about the affliction of peer-review addiction, also termed extreme reviewing, "not because my reviews are extremely negative/positive or extremely long/short, but because I am currently reviewing what I would typically consider (for me) to be an extreme number of manuscripts (8), in addition to a few dozen proposals. Another way to describe this syndrome is that I have become a reviewing doormat, unable to just say no to editor requests to review." Female Science Professor discusses several possible reasons for this behaviour in her post. There is also some discussion in the comments about reasonable reviewing loads: whether for manuscripts submitted to journals, or for grants.

Posted by Maxine Clarke on January 14, 2008
Categories: Perspectives | Permalink | Comments (0)

From a post about peer review at the blog Unqualified Offerings:

"The simple fact of peer review is that it’s actually quite a modest hurdle. All you have to do is find one editor and a couple of reviewers who find the work plausible and well-executed. Once you’ve been on both sides of the process, you realize that it’s just a preliminary quality check, a first pass before it’s put out there for a wider audience. Some laymen seem to attribute too much significance to it, and other laymen seem to recoil against that misperception by concluding that peer review is too weak of a system. The truth is that it’s not supposed to be a stringent filter. It’s just supposed to be a first pass."

Posted by Maxine Clarke on December 18, 2007
Categories: Perspectives | Permalink | Comments (0)

Bloggers for peer-reviewed research reporting, or BPR3, was conceived by scientists and others who write informally about research on the Internet as a simple way to denote that a blog post or website article is discussing peer-reviewed work. Their mission statement: "Bloggers for Peer-Reviewed Research Reporting strives to identify serious academic blog posts about peer-reviewed research by offering an icon and an aggregation site where others can look to find the best academic blogging on the Net."
As well as creating an icon for bloggers to denote when their posts concern peer-reviewed research, the organization will host a central web site where snippets from these posts will be displayed, with links back to the original posts. Readers will be able to choose topics of interest and view only those posts, if they wish.
Eventually, the intention is that bloggers will be able to enter a DOI (digital object identifier) or other unique identifier, and automatically generate code to post the icon, link to the post to the BPR3 site and its aggregation tools, and generate a properly formatted research citation which links to the original article.
Now, the first stage is complete -- the icons have been created (with a lot of helpful input from large numbers of science bloggers). They must be popular, as the site from which people can collect their code has been down for a couple of days due to excessive traffic, presumably. However, it is now up and viewable, so if you want to pick up an icon for your own blog, or just find out more details of the project, please visit this posting. From this post (29 October): "Anyone can use these icons to show when they're making a serious post about peer-reviewed research, rather than just linking to a news article or press release. Within a month, these blog posts will also be aggregated here, so everyone can go to one place to locate the most serious, thoughtful analysis and commentary on the web."
Inevitably, given the engagingly self-referential and gossipy nature of blogging, release of these icons attracted a lot of comment and discussion in the blogosphere. Dave Munger, the driving force behind BPR3, has collected links to all these articles into one post here.
Perhaps in some of this discussion, this question has been addressed already, but as I am afraid I do not have time to read all of the 26 (so far) articles, I raise it here. The icon seems to me an excellent way to indicate that the subject of a blog post is a peer-reviewed research article (so long as there is a mechanism to report abuse and remove the icon from posts that use it incorrectly). And the intentions of BPR3 in providing links to the original research article being discussed are admirable. However, these indicators in themselves do not seem to me necessarily to be an indicator of quality of the blog post itself. On a blog, anyone can write anything about anything, whether or not the topic under discussion is peer-reviewed. A lot of traffic to, or comments on, a blog post is not in itself an indicator of quality. (It could, indeed, be the opposite.)
After the blog posts are themselves linked to the original research article, it will be possible for them to be formally cited, making them eligible for aggregation into citation databases. Blog posts that enter the mainstream scientific debate in this sense would have a quality indicator associated with the post itself rather than the research they describe. Is this the intention?

Posted by Maxine Clarke on November 02, 2007
Categories: Perspectives, Quality and value | Permalink | Comments (4)

Christopher Lee

Boundary-crossing research meets border patrol

Both universities and funding agencies have proclaimed the need for more ‘interdisciplinary’ research and more ‘interdisciplinary’ teams, yet publication of such work raises a unique set of challenges for peer review for which many traditional single-discipline journals are not fully prepared. Many of the most exciting research fields today are themselves hybrids of multiple disciplines, yet the peer review system gives rise to problems in assessing interdisciplinary research. The system can be fixed, given the tremendous opportunities today for new approaches

Continue reading "Perspective: Peer review of interdisciplinary scientific papers" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (2)

Debomoy Lahiri

Peer review would be improved by discussion across the lab

The current individual-based review system has its limitations. Modern research is both multidisciplinary and technical, and it is often difficult to find reviewers that have significant expertise across subfields together with technical know-how in a specific discipline. They are also given only a limited time to complete their review. So reviewers, even those within the same field of research, often differ in their evaluation of a paper. As an alternative, I suggest we use a system which I call peer group review.

Continue reading "Perspective: The case for group review" »

Posted by Maxine Clarke on June 29, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (0)

Tracey Brown

Understanding peer review is key to developing informed opinions about scientific research.

The general public are presented with 'scientific findings' from a wide range of sources, some more credible than others. Educators complain that pupils and students use web research with little regard for the status of what they find. Medical helplines are inundated with calls about risks and cures following media stories. And, much to the frustration of scientists, unwarranted scares, pseudoscience and health fads abound. How can judgements be made and useful questions asked?

Continue reading "Perspective: 'I don't know what to believe'" »

Posted by Maxine Clarke on June 20, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (3)

Thomas DeCoursey

Should authors be told who their reviewers are?

The goal of any change in the peer review system must be to improve the quality of review, where quality is determined by two distinct functions: filtering manuscripts for publication in a given journal; and making constructive suggestions on how the manuscript or study could be improved. Would open review (in which reviewers sign their reviews) accomplish this goal? I have experienced several cases of open review, intentional and unintentional, with mixed results.

Continue reading "Perspective: The pros and cons of open peer review" »

Posted by Maxine Clarke on June 14, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (9) | TrackBacks (1)

John Moore

Even reviewed literature can be cherry-picked to support any argument.

The research community understands that scientific information that has not been peer reviewed should not be taken seriously. As scientists, we discriminate between what is put out on blogs or in press releases and what is published in the formal scientific literature. We also know the difference between a peer-reviewed primary paper or review, and an unreviewed letter to the editor or opinion piece. In other words, we understand the peer-review system, and use it as a filter to sort the wheat from the chaff.

Continue reading "Perspective: Does peer review mean the same to the public as it does to scientists?" »

Posted by Maxine Clarke on June 05, 2006
Categories: Peer review debate, Perspectives | Permalink | Comments (3) | TrackBacks (1)