« May 2009 | Main | August 2009 »

Archive by date: June 2009

Hoax paper accepted for publication


Bookmark in Connotea

A story published in Nature News online on 15 June, describes how the editor-in-chief of a journal is to resign after claiming that the publisher, Bentham Science Publishing, accepted a hoax article for publication without his knowledge.
From the Nature News story:
The fake, computer-generated manuscript was submitted to The Open Information Science Journal by Philip Davis, a graduate student in communication sciences at Cornell University in Ithaca, New York, and Kent Anderson, executive director of international business and product development at The New England Journal of Medicine. They produced the paper using software that generates grammatically correct but nonsensical text, and submitted the manuscript under pseudonyms in late January. After receiving several unsolicited invitations by e-mail to submit papers to open-access journals published by Bentham under the author-pays-for-publication model, Davis wanted to test if the publisher would "accept a completely nonsensical manuscript if the authors were willing to pay". The manuscript was accepted with a request from the publisher for Davis to pay US$800 to its subscriptions department, based in the United Arab Emirates, before the article was published. Davis then retracted the article.
Bambang Parmanto, an information scientist at the University of Pittsburgh, Pennsylvania, and editor-in-chief of The Open Information Science Journal, told Nature that he had not seen the manuscript or any peer review comments before it was accepted. Nor was he informed that the manuscript had been accepted for publication. "I think this is a breach of policy," he says, adding: "I will definitely resign. Normally I see everything that comes through. I don't know why I did not see this. I at least need to see the reviewer's comments." Parmanto says that Bentham Science Publishing told him that the manuscript had been reviewed by one member of the journal's editorial board. "The peer review didn't work," says Parmanto, who now fears that the journal's publishing system could be open to abuse. "The publisher could take advantage of the fees, and that is why I want to leave," he says.
Mahmood Alam, director of publications at Bentham Science Publishing, told Nature in an e-mail statement that "submission of fake manuscripts is a totally unethical activity and must be condemned." He defended Bentham's peer review process, saying, "a rigorous peer review process takes place for all articles that are submitted to us for publication. Our standard policy is that at least two positive comments are required from the referees before an article is accepted for publication." In this particular case, "the paper was reviewed by more than one person".
Peter Suber, a philosopher at Earlham College in Richmond, Indiana, and a proponent of open-access publishing, is worried that the case could turn people against the author-pays open-access model. "There are many legitimate and rigorous open-access journals that use this same business model," he says.
Further details are provided in two articles at The Scholarly Kitchen blog: an account of the experiment by Philip Davis, Nonsense for dollars; and an editorial, The tip of an Iceberg, by Kent Anderson.
Janet Young, in a comment to the Nature News story, writes: "I've had six requests to review papers from a Bentham Science journal over the last year. The first was for a paper in my field, but I refused as I was very busy at the time. The other five have been for papers in fields I know nothing about - I would have been an utterly inappropriate reviewer had I accepted the requests (I didn't). Each request had the full paper attached to the email, rather than just an abstract. That seems like a very unusual review process to me."
See Nature's news website for the full version of the article, and to add your own comments. You are also welcome to comment here.

No gender bias identified in peer-review of grant applications


Bookmark in Connotea

NatureJobs reports on the contentious topic of possible gender bias in peer review (Nature 459, 602; 2009). Peer review assesses what is of value in science, yet it has been criticized for biases. One such perceived bias is gender, although evidence for such a bias has been contradictory. A 2007 meta-analysis (L. Bornmann et al. J. Informet. 1, 226–238; 2007; reported at the time in NatureJobs Nature 445, 566; 2007) concluded that women are at a disadvantage in peer review of funding applications. As this study incorporated all known research on this issue, it was suggested to be definitive.
Nevertheless, H. W. Marsh et al. (Am. Psychol. 63, 160–168; 2008) presented conflicting results the following year. This study was comprehensive, based on data from the Australian Research Council (10,023 reviews by 6,233 external assessors of 2,331 proposals from all disciplines), and concluded that the gender of the applicant had no effect on the outcomes of peer review, irrespective of the discipline, gender and nationality of the reviewers, and whether reviewers were selected by a funding panel or chosen by the applicants.
Why should these two studies have conflicting results? To investigate, Marsh and Bornmann now report in NatureJobs how both research teams worked together to reanalyse the data and extend the original meta-analysis. They describe how they applied new, stronger statistical approaches to 66 sets of results representing 353,725 proposals from 8 countries. In this extended study, which will be published in Review of Educational Research, they found no effect of the applicant's gender on the peer review of their grant proposals. This lack of effect held across country, year of publication of the studies included in the meta-analysis, and disciplines ranging from physical sciences to the humanities.
At least for grant applications, all of the co-authors from each of the research teams agree that the weight of evidence suggests that the applicant's gender has no effect on the outcome of peer review, and that these findings are robust and broadly generalizable.
Herbert Marsh is a professor of education at the University of Oxford, UK
Lutz Bornmann is a PhD student at the ETH University in Zurich, Switzerland.
This Peer-to-Peer post is an edited version of their NatureJobs article (which is free to access online).
There is some discussion of this NatureJobs article and the timing of its publication at Nautilus blog.
Note: readers interested in the Review of Educational Research paper in advance of its publication can obtain a copy directly from Dr Bornmann or from the NatureJobs editors.