Peer-to-peer

The Valve - A Literary Organ | Idea for Discussion: An Academic Blog Review

Amardeep Singh, of The Valve literary review, proposes a system of academic blog reviewing, whereby people self-select individual blog posts they’ve written for review by others, via a community system such as proposed in some of the contributions to Nature's peer-review debate (see left-hand vertical column for a link). The proposal is intended to help those with academic blogs to be able to "publish", or at least, to "credentialize", selected shorter posts in addition to their more formal publications in the literature.

Posted by Maxine Clarke on March 29, 2007
Categories: Systems | Permalink | Comments (0)

Shaping the Message, Distorting the Science - Center for Media and Democracy

Sheldon Rampton, Research Director of the Center for Media and Democracy, is testifying today (28 March) before the US House of Representatives Committee on Science and Technology in a hearing entitled "Shaping the Message, Distorting the Science: Media Strategies to Influence Science Policy." Mr Rampton's written testimony is available here, and a webcast of the hearing is available here. Mr Rampton discusses the reliability of scientific knowledge, and the way in which it has been, and is, used as a tool to manipulate public opinion. He provides some examples of cases in which industries (such as the tobacco and petroleum industries) have undermined the journal peer-review process, and concludes:

"The manipulation of science for public relations or political advantage inevitably has a corrupting effect on science itself. It undermines the integrity and objectivity of scientific research. It creates confusion in the minds of policymakers and the general public. What is needed, therefore, is greater public transparency regarding the sponsorship of science and of organizations that claim to speak on scientific matters. The public and policymakers have a right and to know who is funding research, what strings are attached to that funding, and how it may be affecting the information we use to make decisions—especially decisions on policy matters that affect us all."

Rampton has authored numerous articles, commentaries and books (with John Stauber) on the subject of this testimony ,including Trust Us We're Experts: How Industry Manipulates Science and Gambles With Your Future ; and Toxic Sludge Is Good For You: Lies, Damn Lies and the Public Relations Industry.

See here for the website of "Shaping the message, distorting the science": US House of Representatives.

Posted by Maxine Clarke on March 28, 2007
Categories: Ethics, Policy and Public, Quality and value | Permalink | Comments (0)

Open Call From the Patent Office - washingtonpost.com

The Washingtonpost.com reports (link above) that the US Patent and Trademark Office is starting a pilot project that will not only post patent applications on the Web and invite comments, but also use a community rating system designed to push the most respected comments to the top of the file, for serious consideration by the agency's examiners. The Washingtonpost.com article compares the the system to that used by Wikipedia, the popular user-created online encyclopedia.

The "peer to patent" project starts next week (week of 2 April) with a pilot programme, led by the Institute for Information Law and Policy at New York Law School and the US Patent and Trademark Office. The pilot will initially accept 250 patent applications from companies including IBM, Intel, Oracle, Hewlett-Packard and Microsoft.

Posted by Maxine Clarke on March 28, 2007
Categories: Policy and Public, Systems | Permalink | Comments (0)

From the April editorial in Nature Chemical Biology:

"Although we strive for transparency in scientific publishing, there is an inherent need for opacity in the peer review process so that editors can receive candid information about the technical merit and potential impact of each manuscript. Given the limitation of referee confidentiality, the editorial team at Nature Chemical Biology is committed to making peer review as transparent as possible. Referee selection is a critical element in this process. We request that potential referees notify us of possible conflicts of interest that may affect their ability to provide an objective assessment of a manuscript. When potential conflicts arise, referees may recuse themselves. Conflicts may include competing financial interests related to the science reported in the article or personal or professional relationships with authors that may affect the referee's impartiality. To minimize conflicts in which scientists may be in competition or hold opposing views on a scientific controversy, the editorial team generally honors author requests to exclude particular scientists from the referee pool. Finally, to enhance transparency, Nature Chemical Biology editors communicate with referees after a decision has been sent to authors. In addition to informing referees of our editorial decision, we send copies of all referee comments, while maintaining the confidentiality of all referees, to each reviewer who was consulted on the manuscript."

The complete editorial is at: Nature Chemical Biology 3, 185 (2007).
Nature journals' policies on competing financial interests are explained at the author and referees' website.

Posted by Maxine Clarke on March 20, 2007
Categories: Quality and value | Permalink | Comments (1)

From ABC News: WASHINGTON, March 19, 2007 — - "A top government climate scientist told Congress today that political appointees without scientific backgrounds are corrupting the scientific process and confusing the public by censoring scientists and improperly editing their research on global warming.
"I believe that the nature of these edits is a good part of the reason for why there is a substantial gap between the understanding of global warming by the relevant scientific community and the knowledge of the public and policymakers," said James Hansen, director of NASA's Goddard Institute for Space Studies. "There has been so much doubt cast on our understanding that they think it's still completely up in the air." "
Republican critics strongly disagreed with Dr Hansen's views. The hearing is the latest of several to explore government censorship of climate scientists at NASA and the National Oceanic and Atmospheric Administration (NOAA).
The full ABC News article is available here.

Posted by Maxine Clarke on March 20, 2007
Categories: Policy and Public | Permalink | Comments (0)

The editors of the Annals of Internal Medicine announce a new policy on reproducibility of research findings in the current (20 March) issue of the journal. From the summary:

"A community of scientists arrives at the truth by independently verifying new observations. In this time-honored process, journals serve 2 principal functions: evaluative and editorial. In their evaluative function, they winnow out research that is unlikely to stand up to independent verification; this task is accomplished by peer review. In their editorial function, they try to ensure transparent (by which we mean clear, complete, and unambiguous) and objective descriptions of the research. Both the evaluative and editorial functions go largely unnoticed by the public—the former only draws public attention when a journal publishes fraudulent research."

The article goes on to discuss the existing strategies used by the journal to guard against publication of invalid or biased research, and announces new policies to increase confidence in published research. "Every original research article will include a statement that indicates whether the study protocol, data, or statistical code is available to readers and under what terms authors will share this information. Sharing will not be mandatory, but we will require authors to state whether they are willing to share the protocol, data, or statistical code."

The Nature journals' policy on availablity of materials is described at our Author and Referees' website.

Posted by Maxine Clarke on March 20, 2007
Categories: Ethics | Permalink | Comments (0)

According to an article in Wired news on 14 March, the "$10 billion science publishing industry hasn't heard the last of a bill that would make publicly funded studies available for free. Sen. John Cornyn (R-Texas) has pledged this year to resurrect the Federal Research Public Access Act (S.2695), which would require federally funded research to become publicly available online within six months of being published."

The Wired article refers to a news story in Nature of 24 January, which describes how a group of journal pubishers have employed a public relations firm to "take on" the 'open-access' movement, to help communicate that the issue is not only about preserving profits, as some of the open-access proponents claim. From Wired :

"Our core message is that we believe in the integrity of the peer-review system and the investments in it," said Brian Crawford, chairman of the executive council of the Professional and Scholarly Publishing Division of the Association of American Publishers. "It's inappropriate for the government (to interfere)."

Later in the article: "A few traditional subscription publishers are also experimenting with open access, making more research freely available after a certain amount of time or if authors pay extra fees. Nature, for example, is experimenting with a free journal and allowing some authors to pay to make their findings publicly available. And earlier this month, the Howard Hughes Medical Institute announced it would pay the large Elsevier publishing house to make institute-funded research available to the public for free after six months. The change will take effect on Sept. 1, affecting research published after that date.

"But it is already clear that a journal like Nature would struggle under an open-access business model," said David Hoole, head of brand marketing for the Nature Publishing Group. "We reject 90 percent of the articles we receive, and spreading the cost of peer review over the few authors who do get published would be very unfair (and would probably deter submissions). We [Nature] have approximately 1,000 authors, and 60,000 subscribers [a year]. It seems fairer to spread the costs over the subscribers." "

Posted by Maxine Clarke on March 16, 2007
Categories: Systems | Permalink | Comments (2)

In peer review, not all peers are equal: so writes John Timmer on ars technica, reporting on a limited study of the quality of peer-review on the journal Annals of Emergency Medicine. Although the study was limited to one journal and hence discipline, the dataset consisted of 306 reviewers who had completed more than 3,000 reviews.
"Satisfactory reviews were compared to unsatisfactory ones, and excellent reviews were separately compared to average. For both of these cases, correlations were tested with a number of factors thought to improve reviewing skills (participation in grant reviews, years of experience, prior coursework in critical appraisal, etc.). These factors were tested both individually and in a multivariable analysis, which should eliminate any confounding factors.
In news that may be disturbing for journal editors everywhere, very few factors leapt out as having a consistent and significant correlation with the quality of a review, although some factors did have strong correlations in individual tests. The only positive factors linked to quality of reviews were age (younger reviewers were better) and working at an academic hospital. Ironically, service on an Institutional Review Board, which evaluates and approves experiments on humans, consistently correlated with lower-quality peer reviews. Even these factors, however, were only slightly better than random at predicting review quality."
Dr Timmer concludes: "There are some clear limitations to this study, given that it applies to a single journal with content that's exclusively medical. But the researchers who performed it note that it may be difficult to extend these studies to other fields, as many journals don't even have a mechanism for evaluating or tracking the quality of reviewers. Given the importance of peer review to the entire scientific enterprise, their strongest conclusion is that more needs to be done to track and evaluate the process in order to ensure that the body of published information is as reliable as possible."

Posted by Maxine Clarke on March 09, 2007
Categories: Quality and value | Permalink | Comments (0)

Doug Lederman reports in Inside Higher Ed on competition in US agricultural research. "The notion that competition produces better science is an underlying principle of the American research enterprise — with the possible exception, historically, of agriculture. Just 10 percent of the more than $2.6 billion that the federal government spends annually on agriculture research is awarded through peer review, with the rest flowing to scientists at the U.S. Agriculture Department or to researchers at land-grant colleges through formulas that have gone largely unchanged for decades."
Mr Lederman describes the current Senate hearings, and various attempts, including that of the Create-21 group, to increase the proportion of funding given to competitive research programmes.

Posted by Maxine Clarke on March 09, 2007
Categories: Quality and value | Permalink | Comments (0)

Richard Akerman, in a 2 March post on Science Library Pad entitled "Soft peer review", provides links to some interesting new reports on the subject. One is called "Social software and new opportunities for peer review", via Library 2.0; another is "Soft peer review? Social software and distributed scientific evaluation" via Academic Productivity blog.
Richard is one of the contributors to Nature's web focus on peer review, published last year and archived in full on Peer to Peer (it contains 22 articles, including Richard's, and remains open to comments from the community of peer-reviewers, authors and scientists).
In his Science Library Pad post, Richard concludes: "I do think the debate about "open" peer review vs. traditional peer review is a bit of a red herring, and it very much concerns me when people suggest that open review can replace traditional (or in the language of this posting "hard") peer review. We have already had open peer review for years, it's called preprint feedback, mailing lists, ArXiV, letters to the editor... it's a tremendous addition to, but not replacement for, the rigourous anonymous peer review system needed to provide a publication filter."

Posted by Maxine Clarke on March 06, 2007
Categories: Systems | Permalink | Comments (0)

Thursday, 01 March 2007
ISLAMABAD (Associated Press of Pakistan, APP): The Chairman of the Higher Education Commission, Dr Atta-ur-Rahman, said on Thursday that co-editors of journals should be selected from industrially advanced countries to attract quality papers from potential researchers. Active and eminent editorial boards are essential for the research journals being printed in the country, he said while addressing a meeting of the Editors of HEC recognized journals, organized here by the Quality Assurance Division of HEC.

He said the editorial board of every research journal should be broad-based, and selection of editorial members be made by searching in high quality journals to attract prestigious editorial board members. The Chairman was of the view that database of OIC Standing Committee on Scientific and Technological Cooperation (COMSTECH), which is excellent in current content information, should be used for the purpose as it is available to all the Islamic countries.

Dr Sohail Naqvi, Executive Director HEC asked the editors to launch their own websites for electronic publications. For the purpose, management software for online paper submission and peer review could be provided by HEC, while the editors will be encouraged to communicate through electronic media. He further asked the editors that ethical issues of plagiarism in their journals should be avoided through an undertaking. The editors should come forward with seminars and workshops to be organized at their universities for improvement of the journals, for which HEC would provide the funds, he added.

More then 70 editors of HEC-recognized journals from universities and organizations all over Pakistan attended the meeting. Prof Dr Shamshad Akbar of Government College University, Lahore, in his presentation shared the main features of high-quality national and international journals and their requirements. Prof Waqaruddin Ahmad of HEJ Karachi also made a presentation on “Journal of Chemical Society of Pakistan” published by HEJ, and explained in detail the impact factor system of the Institute of Scientific Information. The participants of the meeting recommended that the HEC should establish operating standards and procedures for research journals.

Posted by Maxine Clarke on March 02, 2007
Categories: Journals | Permalink | Comments (0)