Alternative Peer Review : Quality Management for 21 st Century Scholarship

1. Alternative Peer Review:Quality Management for 21st Century Scholarship http://www.public.iastate.edu/~gerrymck/APR.ppt Gerry McKiernan Science and Technology Librarian and Bibliographer Iowa State University Library Ames IA USA gerrymck@iastate.edu

2. Workshop on Peer Review in the Age of Open Archives International School for Advanced Studies Interdisciplinary Laboratory Trieste, Italy May 23-24, 2003

3. THANK YOU! • Workshop Advisory Board [(Marco Fabbrichesi (INFN/SISSA Italy), Stevan Harnad (University of Southampton, UK), Stefano Mizzaro (University of Udine, Italy) and Corrado Pettenati (CERN Library, Geneva, Switzerland)] • Iowa State University, Faculty Senate, Committee on Recognition and Development • European Commission • Iowa State University Library • Heike Kross, Ph.D.

4. DISCLAIMER (1) The screen prints selected for this presentation are for educational purposes and their inclusion does not constitute an endorsement of an associated product, service, place, or institution.

5. DISCLAIMER (2) The views and opinions expressed in this presentation are those of the presenter and do not constitute an endorsement by Iowa State University or its Library.

6. NOTICE No editors, authors, or referees were harmed in the preparation of this presentation.

7. http://www.sciencecartoonsplus.com/

8. Giuseppe De Nittis (1846-1884) The Macchiaioli / Italian Impressionists

9. Campo di Biche (1875)

10. PEER REVIEW: DEFINITION “Peer review is the assessment by an expert of material submitted for publication.” Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-358.

11. PEER REVIEW: PURPOSES Peer review helps to ensure that published research is: Benefited from guidance by experts Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-358.

12. PEER REVIEW: STRENGTHS The underlying strength of peer review is“…the concerted effort by large numbers of researchers and scholars who work to assure that valid and valuable works are published and conversely to assure that invalid or non-valuable works are not published … .” Anne C. Weller, Editorial Peer Review: Its Strengths and Weaknesses. (Medford, NJ: Information Today, 2001).

13. “Houston, We Have a Problem!”

14. PEER REVIEW: PROBLEMS • Subjectivity • Bias • Abuse • Detecting defects • Fraud and Misconduct • Delay Fytton Rowland, “The Peer-Review Process,” Learned Publishing 15 no. 4 (October 2002): 247-258. Report version: http://www.jisc.ac.uk/uploaded_documents/rowland.pdf

15. SUBJECTIVITY • Summary rejections by editor without sending the paper to referees • Choice of referees by the editor (choosing for example, a known harsh referee for a paper the editor wishes to see rejected)

16. BIAS • Discrimination against authors because of their nationality, native language, gender or host institution • Situations where author and referee are competitors in some sense, or belong to warring schools of thought

17. ABUSE • Too many articles out of one piece of research, or duplicate publication • Intellectual theft: omission or downgrading of junior staff by senior authors • Plagiarism (stealing others yet unpublished work that has been sent for review) • Delaying publication of potentially competing research

18. DETECTING DEFECTS • Identification of factual errors within submission

19. FRAUD and MISCONDUCT • Fabrication of results • Falsification of data • False claim of authorship for results

20. DELAY “There is much muttering about publication delay, a real enough problem, especially in paper publication, but peer review itself is often responsible for as much of the delay as the paper publication and distribution process itself.” Stevan Harnad Stevan Harnad, “Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals, in Scholarly Publication: The Electronic Frontier, edited by Robin P. Peek and Gregory B. Newby (Cambridge MA: MIT Press, 1996). http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html

21. http://www-marketing.wharton.upenn.edu/ideas/pdf/peerpaper.pdfhttp://www-marketing.wharton.upenn.edu/ideas/pdf/peerpaper.pdf

22. “Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless in detecting fraud.” Richard SmithEditor, BMJ Richard Smith, “Opening Up BMJ Peer Review,” BMJ 318 (7175) (January 2 1999): 4-5

23. Stephen Lock, A Difficult Balance: Editorial Peer Review in Medicine (Philadelphia, PA: ISI Press, 1986).

25. EXAMPLE JanHendrikSchön

26. - http://archive.salon.com/tech/feature/2002/09/16/physics/

27. Science Nature http://physicsweb.org/article/news/6/9/15/

28. http://agenda.cern.ch/fullAgenda.php?ida=a01193

29. RECOMMENDATIONSWorkshop on the Open Archives Initiative (OAI)and Peer Review Journals in Europe, CERN, Geneva Switzerland, March 22-24, 2001 “The participants were unanimous in their belief that the certification of scholarly work remains a fundamental part of a system for scholarly communication.” “It was [also] generally believed that the electronic environment allows for novel approaches to accord a stamp of quality to scholarly works.” Alison Buckholtz, Raf Dekeyser, Melissa Hagemann, Thomas Krichel, and Herbert Van de Sompel, “Open Access: Restoring Scientific Communication to Its Rightful Owners,” European Science Foundation Policy Briefing 21 (April 2003): 1-8. http://www.arl.org/sparc/SPB21_OAI.pdf

30. “Let us be imaginative in exploring the remarkable possibilities of this brave new medium.” Stevan Harnad, “Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals, in Scholarly Publication: The Electronic Frontier, edited by Robin P. Peek and Gregory B. Newby (Cambridge MA: MIT Press, 1996). http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html

35. “Let us be more imaginative in exploring the remarkable possibilities of this brave new medium.”With Apologies to Stevan Harnad

37. TM http://lockss.stanford.edu/

38. LOCKSS For centuries libraries and publishers have had stable roles: publishers produced information; libraries kept it safe for reader access. There is no fundamental reason for the online environment to force institutions to abandon these roles. • The LOCKSSmodel capitalizes on the traditional roles of libraries and publishers.  LOCKSS creates low-cost, persistent digital "caches" of authoritative versions of http-delivered content. 

39. LOCKSS • The LOCKSSsoftware enables institutions to locally collect, store, preserve, and archive authorized content thus safeguarding their community's access to that content.  • The LOCKSS model enforces the publisher's access control systems and, for many publishers, does no harm to their business models.

40. LAMPSS Lots of Alternative Models Provide Sensible Solutions

42. DISCLAIMER ON The alternative peer review models profiled are for informational and educational purposes only and do not necessarily constitute an endorsement.

43. Neo-Classical Certification-based Open Peer Review Commentary-based Collaborately-filtered Institution-based Citation-based Index-based Metadata-based Computer-assisted NO Peer Review Moderator-based Tier-based ALTERNATIVE PEER REVIEW

44. NEO-CLASSICAL PEER REVIEW

45. Neo-Classical Peer Review

46. Neo-Classical Peer Review

47. Neo-Classical Peer Review

48. CERTIFICATION-BASED

49. Certification-Based “The process of pre-publication peer review could be improved and become a more reliable indicator of manuscript quality if reviewers were trained in, and subsequently applied systematically, critical skills and use of a hierarchy of evidence to classify submitted articles being reviewed.” Stephen Pritchard , “Peer Review - a Proposal for Change,” Paper presented at Thinking Globally - Acting Locally: Medical Libraries at the Turn of an Era, 8th European Conference of Health and Medical Libraries, September 16-21, 2002, Deutsche Zentralbibliothek für Medizin, Köln, Germany. http://www.zbmed.de/eahil2002/abstracts/pritchard.pdf

50. OPEN PEER REVIEW