Peer-to-peer

Dennis McDonald - Managing Technology - A Comparison of Blogging and Journal Peer Review
More comparisons between journal peer-review and blogging.
In these two posts on his blog All Kind Food, technology consultant Dennis McDonald attempts to encapsulate the key features that distinguish peer-reviewed journals from blogs. It is an ambitious challenge, to compare the formal structure of journal publication with the "anything goes" nature of the blogosphere, where last week might as well be last century.
Nevertheless, Mr McDonald has identified some key similarities: the motivation of prestige and reputation (journal impact measures or blog rankings in indexes like Technorati); "secondary publishing" via abstracting and indexing (journals) or the equivalent for blogs (Connotea, Delicious, Technorati listings); citations (formal references for journals or links and trackbacks for blogs), and various types of social network.
Although the publishing processes may be similar, blogs are not peer-reviewed. While acknowledging this crucial difference, Mr McDonald believes that it is these social networking aspects of both media where the peer-review process could integrate with the blogging system:
"The information contained in a refereed and published journal article, for example, may have been previously disseminated to other researchers via email, workshops, conferences or conference proceedings, or final reports submitted to funding agencies. In other words, the information that appears in a refereed journal article may have long been communicated to members of the author's existing social and professional networks, especially to those working in the same or in very closely related technical or professional areas. It's not unusual, for example, that research reported in a journal article has been long since superseded by other work done by the author.
The article itself, while it now becomes available to a much wider audience through the a wide range of physical and electronic access channels, acts not only as a conduit of research information but also an advertisement of the skill and accomplishments of the author, filtered by the "halo effect" of prestige and recognition of the journal in which it is published."

Posted by Maxine Clarke on January 30, 2007
Categories: Systems | Permalink | Comments (0)

Q. Dear Nature Editors,
I am an editorial assistant for a journal published by the American Educational Research Association, which has just changed editorial teams.The current team is interested in improving the quality of the journal. Your journal was selected because of it's caliber. We are particularly interested in knowing if you have guidelines that manuscript reviewers are asked to follow. If you do have guidelines for your reviewers,we would like to request a copy.

Thank you for your time,

Editorial Assistant

A. Dear Dr Editorial Assistant

Thank you for your message. Yes, the Nature journals have reviewer guidelines and a peer-review policy. You can access all relevant information on our peer-review web page.
Please let me know if, having read this page and the associated pages, you have any more questions. These pages are public information; you are welcome to copy and redistribute them so long as you cite the source (eg the URL).

Best wishes

Maxine Clarke
Nature
www.nature.com/nature/authors/

Posted by Maxine Clarke on January 29, 2007
Categories: Peer Review Feedback | Permalink | Comments (0)

In their Correspondence "Journals should set a new standard in transparency", published in today's Nature, Robert P. Dellavalle, Kristy Lundahl, Scott R. Freeman and Lisa M. Schilling write:

We applaud your commitment, as expressed in the Editorial "Peer review and fraud" (Nature 444, 971–972; doi:10.1038/444971b 2006), to raising peer-reviewer awareness about detecting fraud. For studies involving humans, independent research ethics committees (in the United States, institutional review boards) provide the first independent critical scrutiny of research protocols. We recently examined the instructions to authors of 103 medical journals and found that none requires authors to provide to readers (as online supplementary information accompanying the publication) the protocols approved by these committees.

As concern increases about the integrity of published scientific research, we believe that biomedical journals should establish a new standard in human-research transparency. They should require authors to state at submission — and, where judged necessary, in their published articles — that the research has been approved by the relevant ethical committees. All journals publishing research on non-human animals ("Animal experiments under fire for poor design" Nature 444, 981; 2006) should do the same for non-human animal protocols.

Journals should also require authors to provide the full protocols approved by these committees for the editors and peer reviewers, and to allow the journal, if it wishes, to publish these protocols as online supplementary information accompanying publication of the main paper.
From: Nature 445, 364 (25 January 2007) | doi:10.1038/445364a; Published online 24 January 2007
We welcome comments on this proposal.

Posted by Maxine Clarke on January 25, 2007
Categories: Ethics | Permalink | Comments (1)

The Voice of San Diego features an interesting article on the role of peer-review in earmarked government-funded projects. The article is about a proposal to build two new dams in California, but it addresses the more general question of the usefulness of earmaking federal money to particular academic institutions for specific research. According to the Voice of San Diego, 10 per cent of the US$30 billion "pork barrel" annual spend now goes to universities.

Critics of the process say that the earmarking diverts funding from projects that have been rigoriously peer-reviewed by expert panels -- not least when institutions likely to benefit spend money on Washington lobbyists. The recipient insitutions, naturally, beg to differ, as the money can mean a new lab or high-tech equipment. An interesting debate follows on where to draw the line between scientific scrutiny and "obvious public good".

Posted by Maxine Clarke on January 18, 2007
Categories: Policy and Public | Permalink | Comments (0)

Andrew Dessler, in his Gristmill blog, reports on yesterday's (17 January) Capitol Hill briefing on science and politics. From Mr Dessler's report:

"First, I argued that the scientific assessment process is the best way to determine what the scientific community thinks about a particular scientific issue.

The key to my argument is that credible scientific advice emerges from a credible process. Scientific results gain credibility by passing peer review, and then being re-tested and multiply verified by the scientific community. In that way, hypotheses are converted into "facts." Scientific advice to policymakers gains credibility by relying on peer-reviewed analyses and then going through multiple levels of peer-review -- as the Intergovernmental Panel on Climate Change reports do.

As a result, the IPCC reports are gold-standard statements of what the scientific community knows about the climate and how confidently we know it.

The worst way to determine what the science tells us, as evidenced by Inhofe's last stand, is a Congressional hearing. There is no guarantee that what the "scientists" at those hearings say is true. There's no peer review of any statement, no fact checking -- it's a free-for-all. If you don't believe me, take a look at some of the statements trotted out by David Deming and Bob Carter. They are just flat-out wrong."

Posted by Maxine Clarke on January 18, 2007
Categories: Policy and Public | Permalink | Comments (0)

Cognitive Daily: Nature's report on open peer review

I'm a bit behind on writing posts because of the launch of the author and referee's website earlier this week (www.nature.com/authors) Somewhat belatedly, therefore, but none the worse for it, I'm drawing attention to a post on Cognitive Daily (link at top of this post) about Nature's peer-review trial, and about the odds for open peer review in general.

In his post, Dave Munger writes: "What kind of incentives would work? The most obvious would be career incentives: if work as a reviewer was rewarded with tenure and promotion, it would soon become one of the top priorities of any scholar. Unfortunately this revolutionary change in the glacial world of academia is about as likely as PZ Myers undergoing a religious conversion, so we probably will need to look elsewhere. Many journals already require authors to review articles as a condition of submitting articles for publication. Perhaps this sort of incentive could be adapted to an open review process. Even so, it would be difficult to administer. How would reviewers be evaluated? By authors? But then wouldn't there be an incentive for reviewers to rubber-stamp articles for publication? For now, it appears that the peer review process as it stands might be the lesser evil."

Dave concludes that some combination of wikis and blogs might one day partially replace traditional peer reivew. But, he says, "Only when contributing to these resources becomes part of the tenure rewards system are they likely to become important factors in the world of academic publishing."

Posted by Maxine Clarke on January 13, 2007
Categories: Systems | Permalink | Comments (0)

Nature Publishing Group has this week launched a website dedicated to authors and peer-reviwers. Authors & Referees @ npg (http://www.nature.com/authors) provides links to and information about the Nature journals' publication policies for primary research papers; author and referee services; and author benefits. We also provide feedback links so potential authors, peer-reviewers, scientists and other readers can ask questions about these topics. This blog (Peer to Peer) and our authors' blog (Nautilus) are integrated into the site.

Find out more about the site here or via the links in the vertical column to the left of this page. We hope to be hearing from you about what you'd like us to include in this site, and receiving your questions and comments about our policies, services and benefits for authors and reviwers.
(This post has also been published on the Nautilus blog.)

Posted by Maxine Clarke on January 12, 2007
Categories: General | Permalink | Comments (1)