1 / 208

Alternative Peer Review : Quality Management for 21 st Century Scholarship

Alternative Peer Review : Quality Management for 21 st Century Scholarship. http://www.public.iastate.edu/~gerrymck/APR.ppt. Gerry McKiernan Science and Technology Librarian and Bibliographer Iowa State University Library Ames IA USA. gerrymck@iastate.edu.

jodis
Download Presentation

Alternative Peer Review : Quality Management for 21 st Century Scholarship

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Alternative Peer Review:Quality Management for 21st Century Scholarship http://www.public.iastate.edu/~gerrymck/APR.ppt Gerry McKiernan Science and Technology Librarian and Bibliographer Iowa State University Library Ames IA USA gerrymck@iastate.edu

  2. Workshop on Peer Review in the Age of Open Archives International School for Advanced Studies Interdisciplinary Laboratory Trieste, Italy May 23-24, 2003

  3. THANK YOU! • Workshop Advisory Board [(Marco Fabbrichesi (INFN/SISSA Italy), Stevan Harnad (University of Southampton, UK), Stefano Mizzaro (University of Udine, Italy) and Corrado Pettenati (CERN Library, Geneva, Switzerland)] • Iowa State University, Faculty Senate, Committee on Recognition and Development • European Commission • Iowa State University Library • Heike Kross, Ph.D.

  4. DISCLAIMER (1) The screen prints selected for this presentation are for educational purposes and their inclusion does not constitute an endorsement of an associated product, service, place, or institution.

  5. DISCLAIMER (2) The views and opinions expressed in this presentation are those of the presenter and do not constitute an endorsement by Iowa State University or its Library.

  6. NOTICE No editors, authors, or referees were harmed in the preparation of this presentation.

  7. http://www.sciencecartoonsplus.com/

  8. Giuseppe De Nittis (1846-1884) The Macchiaioli / Italian Impressionists

  9. Campo di Biche (1875)

  10. PEER REVIEW: DEFINITION “Peer review is the assessment by an expert of material submitted for publication.” Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-358.

  11. PEER REVIEW: PURPOSES Peer review helps to ensure that published research is: Benefited from guidance by experts Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-358.

  12. PEER REVIEW: STRENGTHS The underlying strength of peer review is“…the concerted effort by large numbers of researchers and scholars who work to assure that valid and valuable works are published and conversely to assure that invalid or non-valuable works are not published … .” Anne C. Weller, Editorial Peer Review: Its Strengths and Weaknesses. (Medford, NJ: Information Today, 2001).

  13. “Houston, We Have a Problem!”

  14. PEER REVIEW: PROBLEMS • Subjectivity • Bias • Abuse • Detecting defects • Fraud and Misconduct • Delay Fytton Rowland, “The Peer-Review Process,” Learned Publishing 15 no. 4 (October 2002): 247-258. Report version: http://www.jisc.ac.uk/uploaded_documents/rowland.pdf

  15. SUBJECTIVITY • Summary rejections by editor without sending the paper to referees • Choice of referees by the editor (choosing for example, a known harsh referee for a paper the editor wishes to see rejected)

  16. BIAS • Discrimination against authors because of their nationality, native language, gender or host institution • Situations where author and referee are competitors in some sense, or belong to warring schools of thought

  17. ABUSE • Too many articles out of one piece of research, or duplicate publication • Intellectual theft: omission or downgrading of junior staff by senior authors • Plagiarism (stealing others yet unpublished work that has been sent for review) • Delaying publication of potentially competing research

  18. DETECTING DEFECTS • Identification of factual errors within submission

  19. FRAUD and MISCONDUCT • Fabrication of results • Falsification of data • False claim of authorship for results

  20. DELAY “There is much muttering about publication delay, a real enough problem, especially in paper publication, but peer review itself is often responsible for as much of the delay as the paper publication and distribution process itself.” Stevan Harnad Stevan Harnad, “Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals, in Scholarly Publication: The Electronic Frontier, edited by Robin P. Peek and Gregory B. Newby (Cambridge MA: MIT Press, 1996). http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html

  21. http://www-marketing.wharton.upenn.edu/ideas/pdf/peerpaper.pdfhttp://www-marketing.wharton.upenn.edu/ideas/pdf/peerpaper.pdf

  22. “Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless in detecting fraud.” Richard SmithEditor, BMJ Richard Smith, “Opening Up BMJ Peer Review,” BMJ 318 (7175) (January 2 1999): 4-5

  23. Stephen Lock, A Difficult Balance: Editorial Peer Review in Medicine (Philadelphia, PA: ISI Press, 1986).

  24. EXAMPLE JanHendrikSchön

  25. - http://archive.salon.com/tech/feature/2002/09/16/physics/

  26. Science Nature http://physicsweb.org/article/news/6/9/15/

  27. http://agenda.cern.ch/fullAgenda.php?ida=a01193

  28. RECOMMENDATIONSWorkshop on the Open Archives Initiative (OAI)and Peer Review Journals in Europe, CERN, Geneva Switzerland, March 22-24, 2001 “The participants were unanimous in their belief that the certification of scholarly work remains a fundamental part of a system for scholarly communication.” “It was [also] generally believed that the electronic environment allows for novel approaches to accord a stamp of quality to scholarly works.” Alison Buckholtz, Raf Dekeyser, Melissa Hagemann, Thomas Krichel, and Herbert Van de Sompel, “Open Access: Restoring Scientific Communication to Its Rightful Owners,” European Science Foundation Policy Briefing 21 (April 2003): 1-8. http://www.arl.org/sparc/SPB21_OAI.pdf

  29. “Let us be imaginative in exploring the remarkable possibilities of this brave new medium.” Stevan Harnad, “Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals, in Scholarly Publication: The Electronic Frontier, edited by Robin P. Peek and Gregory B. Newby (Cambridge MA: MIT Press, 1996). http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html

  30. “Let us be more imaginative in exploring the remarkable possibilities of this brave new medium.”With Apologies to Stevan Harnad

  31. TM http://lockss.stanford.edu/

  32. LOCKSS For centuries libraries and publishers have had stable roles: publishers produced information; libraries kept it safe for reader access. There is no fundamental reason for the online environment to force institutions to abandon these roles. • The LOCKSSmodel capitalizes on the traditional roles of libraries and publishers.  LOCKSS creates low-cost, persistent digital "caches" of authoritative versions of http-delivered content. 

  33. LOCKSS • The LOCKSSsoftware enables institutions to locally collect, store, preserve, and archive authorized content thus safeguarding their community's access to that content.  • The LOCKSS model enforces the publisher's access control systems and, for many publishers, does no harm to their business models.

  34. LAMPSS Lots of Alternative Models Provide Sensible Solutions

  35. DISCLAIMER ON The alternative peer review models profiled are for informational and educational purposes only and do not necessarily constitute an endorsement.

  36. Neo-Classical Certification-based Open Peer Review Commentary-based Collaborately-filtered Institution-based Citation-based Index-based Metadata-based Computer-assisted NO Peer Review Moderator-based Tier-based ALTERNATIVE PEER REVIEW

  37. NEO-CLASSICAL PEER REVIEW

  38. Neo-Classical Peer Review

  39. Neo-Classical Peer Review

  40. Neo-Classical Peer Review

  41. CERTIFICATION-BASED

  42. Certification-Based “The process of pre-publication peer review could be improved and become a more reliable indicator of manuscript quality if reviewers were trained in, and subsequently applied systematically, critical skills and use of a hierarchy of evidence to classify submitted articles being reviewed.” Stephen Pritchard , “Peer Review - a Proposal for Change,” Paper presented at Thinking Globally - Acting Locally: Medical Libraries at the Turn of an Era, 8th European Conference of Health and Medical Libraries, September 16-21, 2002, Deutsche Zentralbibliothek für Medizin, Köln, Germany. http://www.zbmed.de/eahil2002/abstracts/pritchard.pdf

  43. OPEN PEER REVIEW

More Related