What is peer review and what is it for?

1. Slides adapted from Dr. Emilio Bruna, Department of Wildlife Ecology and Conservation, University of Florida

2. What is peer review and what is it for? Peer review: System by which research is scrutinized by independent experts PR used for three major purposes: • Allocation of research funding • Publication of research in scientific journals • Assessing research and education of institutions

3. Peer review has two main functions 1) Technical evaluation: Is the science sound? 2) Subjective evaluation: Is the science interesting and important? Where did all the permafrost go?

4. Strengths of Peer Review • PR improves the quality of research • PR improves the presentation of research findings • PR helps eliminate fraud and weeds out poor quality research Hwang Woo-suk • Anonymity fosters criticism • Independence of reviewers can reduce cronyism

5. Potential problems with peer review 1) Bias 1) Gender Bias 2) Institutional Bias 3) Geographic Bias 4) Bias against negative results (pressure for higher citation rates) 5) Bias in favor of established researchers 6) Bias against novel ideas (or results that contradict the reviewers)

6. Case Study in Gender Bias • Women: 44% of biomed phd’s, but: • only 25% of the postdocs • only 7% of professional positions. To help understand why, Wenneras and Wold (1997, Nature) used data from applications for the Medical Research Council Fellowships. Success rates for women less than 50% that of men

7. Applicants submit: • CV, bibliography, and proposal • reviewers assign score of 0-4 for: • (a) competence, (b) relevance of research, and (c) methods • 3) Final score = 0-64 (multiply a x b x c) Wenneras and Wold compared scores of males and females in 3 categories and analyzed the factors responsible for the differences

8. Gender Bias: Results • males • females Competence Methodology Relevance • 2.46 • 2.21 • 2.54 • 2.37 • 2.62 • 2.49 Total Score 17 13.8 Why is the competency score so low? Is women’s work of lower quality? Wenneras and Wold 1997(Nature)

9. Gender Bias: Results W&W calclulated “index of productivity” based on: 1) number of publications 2) number of first author publications, and 3) Journal impact fator 4) Citation rates Still, maybe women are working in areas that are less “interesting”, collaborate less, or are based at lower-quality institutions... From Wenneras and Wold 1997(Nature)

10. Gender Bias: Results Used multiple regression to evaluate the effects of: gender, nationality, field in which degree was awarded, field of research, university affiliation, review committee to which the proposal was submitted, location of postoc, recommendation letter, affiliation with memeber of the review committee Two factors were of overwhelming significance: Gender: to receive the same “competence score” as a male colleague, women need to exceed him by 64 impact points. 64 pts = 3 papers in Science/Nature or 20 in top specialized journals Affiliation: affiliation with review committee member = 67 impact points. Note the potential synergistic effects of gender and affiliation....

11. Potential problems with peer review (cont.) 2) Inefficient 1) Difficulty in recruiting and retaining referees 2) Failure to meet deadlines 3) Inconsistency between referees http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003202

12. When you review…. • Declare conflicts of interest before your start • Summarize the main points to assure the author that you understood it • Begin by emphasizing the positive • Give constructive and specific suggestions for improving the analyses and interpretation of results, structure, content and writing • Be polite, even when the author has made basic mistakes • Help the editor Ad hominem attacks are NEVER appropriate