Effects of reviewers' gender on assessments of a gender-related standardized manuscript.

BACKGROUND: Several factors including reviewers' gender can influence the assessment of submitted manuscripts. PURPOSE: This study was designed to investigate the effects of reviewers' gender on the assessment of a gender-related study. METHOD: One hundred reviewers (50 women) were randomly selected from Academic Medicine's reviewer pool. Two versions of an empirical study of medical students in which women forecast a lower professional income than men were prepared, and each version was randomly assigned to 25 men and 25 women reviewers. The two versions were identical except for one sentence in the abstract and two sentences in the conclusion. Version 1 attributed the lower forecast income of women to intrinsic gender factors (e.g., lower financial incentives). Version 2 attributed the difference to extrinsic social learning factors (e.g., socialization bias). RESULTS: Results showed no significant gender difference in reviewers' evaluations of the two manuscript versions. CONCLUSION: Although no evidence of reviewers' gender bias was found in this study, alternative explanations of the findings suggest that further empirical research is needed.

Roles for scientific societies in promoting integrity in publication ethics.

Scientific societies can have a powerful influence on the professional lives of scientists. Using this influence, they have a responsibility to make long-term commitments and investments in promoting integrity in publication, just as in other areas of research ethics. Concepts that can inform the thinking and activities of scientific societies with regard to publication ethics are: the "hidden curriculum" (the message of actions rather than formal statements), a fresh look at the components of acting with integrity, deviancy as a normally occurring phenomenon in human society, and the scientific community as an actual community. A society's first step is to decide what values it will promote, within the framework of present-day standards of good conduct of science and given the society's history and traditions. The society then must create educational programs that serve members across their careers. Scientific societies must take seriously the implications of the problem; set policies and standards for publication ethics for their members; educate about and enforce the standards; bring the issues before the members early and often; and maintain continuing dialogue with editors.

Impartial judgment by the "gatekeepers" of science: fallibility and accountability in the peer review process.

High publication demands and the low acceptance rate of peer review journals place the journal editors and their reviewers in a powerful position. Journal reviewers have a vital role not only in influencing the journal editor's publication decisions, but also in the very nature and direction of scientific research. Because of their influence in peer review outcomes, journal reviewers are aptly described as the "gatekeepers of science." In this article we describe several pitfalls that can impede reviewers' impartial judgement. These include such issues as confirmatory bias, the negative results bias (the file drawer problem), the Matthew effect, the Doctor Fox effect, and gender, race, theoretical orientation, and "political correctness." We argue that procedures currently used by many professional journals, such as blind or masked review, may not completely alleviate the effects of these pitfalls. Instead, we suggest that increasing reviewers' awareness of the pitfalls, accountability, and vigilance can improve fairness in the peer review process. The ultimate responsibilities belong to the journal editors who are confronted with the difficult task of satisfying journal readers, contributors, reviewers, and owners. We recommend that the journal editors conduct periodic internal and external evaluations of their journals' peer review process and outcomes, with participation of reviewers, contributors, readers and owners.