1 / 21

The Blind Leading the Blind

Outline. The five best features of double-blind reviewingBeyond double-blind. Feature

rhea
Download Presentation

The Blind Leading the Blind

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. The Blind Leading the Blind Prof. Short* Dr. Tall* * Names obfuscated and affiliations omitted to preserve anonymity

    2. Outline The five best features of double-blind reviewing Beyond double-blind

    3. Feature #1 Enables useful feedback on half-baked papers, without fear of embarrassment (keeping those under-worked PC members busy)

    4. Feature #2 Slows the advancement of science to a manageable pace, by eliminating rapid dissemination of results

    5. Feature #3 Allows job-seeking PhD students to allude to spectacular new results, which unfortunately they can’t talk about (regardless of whether the results actually exist)

    6. Feature #4 Discourages those annoying high-impact projects with recognizable names and many-author papers that build on one another

    7. Feature #5 Facilitates “flow” of ideas from authors to reviewers (without the irritating requirement of attribution)

    8. But… Double-blind doesn’t go nearly far enough

    9. Problem #1 Senior reviewers can intimidate the junior reviewers of a paper during discussions

    10. Problem #1 Senior reviewers can intimidate the junior reviewers of a paper during discussions SOLUTION: Triple-Blind Reviewers don’t know who the other reviewers are

    11. Problem #1 Senior reviewers can intimidate the junior reviewers of a paper during discussions SOLUTION: Triple-Blind Reviewers don’t know who the other reviewers are This one is real!

    12. Problem #2 Authors of high-impact papers become more famous than authors of insignificant papers

    13. Problem #2 Authors of high-impact papers become more famous than authors of insignificant papers SOLUTION: Quadruple-Blind Authors of published papers are anonymous

    14. Problem #2 Authors of high-impact papers become more famous than authors of insignificant papers SOLUTION: Quadruple-Blind Authors of published papers are anonymous Someone (perhaps Jim Gray...) was 20 years ahead of his or her time with the 1985 “Anon et al.” benchmarking paper

    15. Problem #3 System is biased in favor of authors who give great talks about their results

    16. Problem #3 System is biased in favor of authors who give great talks about their results SOLUTION: Quintuple-Blind PC chair gives all the talks

    17. Problem #4 Famous researchers decline to serve on PCs for second-tier conferences

    18. Problem #4 Famous researchers decline to serve on PCs for second-tier conferences SOLUTION: Sextuple-Blind Conferences are anonymous — PC members don’t know what conference they’re agreeing to review for

    19. Problem #5 Researchers insist on sending their best work to the best conferences, which is unfair to second-rate venues

    20. Problem #5 Researchers insist on sending their best work to the best conferences, which is unfair to second-rate venues SOLUTION: Septuple-Blind Conference submissions are picked randomly from a global pool

    21. Acknowledgements Thanks to … An anonymous west-coast professor with a photography habit An anonymous Midwest professor with the same first name as his (or her!) advisor from whom we “borrowed” some of these ideas (when they weren’t looking)

More Related