No demonstrated gender bias in double-blind peer review
The Editorial 'Working double-blind' (Nature 451, 605–606; 2008), also republished on this blog and stimulating more than 70 comments, referred to a study (1) that found more female first-author papers were published using a double-blind, rather than a single-blind, peer-review system. The data reported in ref. 1 have now been re-examined (2). The conclusion of ref. 1, that Behavioral Ecology published more papers with female first authors after switching to a double-blind peer-review system, is not in dispute. However, ref. 2 reports that other similar ecology journals that have single-blind peer-review systems also increased in female first-author papers over the same time period. After re-examining the analyses, Nature has concluded that ref. 1 can no longer be said to offer compelling evidence of a role for gender bias in single-blind peer review. In addition, upon closer examination of the papers listed in PubMed on gender bias and peer review, we cannot find other strong studies that support this claim. Thus, we no longer stand by the statement in the fourth paragraph of the Editorial, that double-blind peer review reduces bias against authors with female first names.
References
1. Budden, A. E. et al . Trends Ecol. Evol. 23, 4–6 (2008).
2. Webb, T. J. , O'Hara, B. & Freckleton, R. P. Trends Ecol. Evol. doi:10.1016/j.tree.2008.03.003 (2008).
Comments
Posted on behalf of: S. J. Melles
Why not also cite the follow up article by Budden et al. 2008 TREE as they state that “The increase in female first authors within Behavioural Ecology (BE) was 7.9% whereas the mean of the other journals was 3.7% +/- 2.1 SD. So, BE falls on the 95% confidence interval (0.6%–7.9%).” Perhaps the jury is still out because there is not enough data and a lack of incentive for journals to make a change and see what happens! I am unconvinced by the results presented in Webb et al. 2008 TREE. They suggest that there was no significant interaction between gender and review type (-0.15 +/- 0.102, P = 0.134), indicating that the increase in female authorship over time in BE is not exceptionally different from the changes in the other journals in the field). What is their sample size? Do they really have sufficient power to detect a significant interaction effect with these data?
In my opinion as long as there is a chance that a few scientists (e.g., Prof. P. Kantorek, of Ryerson University in Toronto Canada, see his blog post on a Nature blog below) are apt to complete single-blind reviews from a gender-biased or racist perspective, then a FAIR, SCIENTIFIC and “OBJECTIVE” system should work to prevent this: A double-blind peer review system prevents this type of bias because even if the reviewer guesses authorship, they can never be 100% certain.
http://www.nature.com/news/2008/080423/full/452918a.html
“My experience as a physicist working with the occasional female colleagues leads me to a subjective impression that women really think differently. Female thinking seems to be more lateral then vertical. By that I mean, women in physics are generally harder working than male colleagues and are great co-workers in terms of encouragement, diligence, and backup support. They do not, however, contribute a great deal of original ideas and rigorous logical analysis to the research. Female judgment seems to more emotionally biased.”
P. Kantorek, posted 23 Apr, 2008
Posted by: S. J. Melles | June 6, 2008 12:34 PM
As a follow up to my last comment, perhaps the movement towards a double-blind system has to start from the bottom up?
I call on the 71% of researchers that have confidence in double-blind peer review and the 56% that prefer it to other forms of review".
Simply refuse to review any more articles from journals that do not maintain a double-blind review system. Why continue to volunteer time to a system that you don't entirely believe in?
Posted by: S. J. Melles | June 6, 2008 12:54 PM
Posted by: Maxine | June 9, 2008 09:18 AM
To Dr Melles:
Your "P. Kantorek" point: journals use several peer-reviewers per paper, and papers go through several rounds of review before publication. The journal editors, together with the several perspectives of independent peer-reviwers, weed-out biased arguments (whatever type of bias). This process would occur whether the journal operated a single or double-blind system.
Your Webb et al. point: these authors reanalysed the journal sample used by Budden et al., as explained in their paper. The questions you raise about sample size and so on apply to the Budden et al. paper as much as to the Webb et al. paper (both of which were published in a peer-reviewed journal).
Posted by: Maxine | June 9, 2008 11:01 AM