1 / 25

Faculty Peer Review Rubric for Online Courses: A Delphi Study

Faculty Peer Review Rubric for Online Courses: A Delphi Study. Steven G. Lesh, Southwest Baptist University Joan D. McMahon, Towson University M. H. N. Tabrizi, East Carolina University Marj Ashcraft, Towson University. The 9 th Sloan-C International Conference on Asynchronous Learning Networks

tiva
Download Presentation

Faculty Peer Review Rubric for Online Courses: A Delphi Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Faculty Peer Review Rubric for Online Courses: A Delphi Study Steven G. Lesh, Southwest Baptist UniversityJoan D. McMahon, Towson UniversityM. H. N. Tabrizi, East Carolina UniversityMarj Ashcraft, Towson University The 9th Sloan-C International Conference on Asynchronous Learning Networks Orlando, Florida November 16, 2003

  2. Alumni Development Grant

  3. Purpose • The purpose of this presentation is to report the results of a four round Delphi study which produced a rubric for use as a peer review faculty assessment tool.

  4. Objectives After completing this mini-course, the learner will be able to: • relate the controversies surrounding the assessment of faculty leading online courses through traditional promotion and tenure guidelines, • describe the methodology of the Delphi technique to produce a peer review faculty assessment rubric, • discuss the review criteria of the rubric, and • review an online course using the grading criteria established in the faculty peer review rubric.

  5. Background • Professional concerns appear in the literature and scholarly forums about faculty delivering online education including: • promotion • tenure • work loads • release time • compensation

  6. Problem • Online courses demand unique pedagogy and communication strategies. However, faculty who lead online courses are being evaluated using more traditional promotion and tenure guidelines.

  7. Solution • A Delphi study was commissioned by online educators to creatively explore ideas, attitudes, and future implications related to peer review of faculty teaching online.

  8. The Delphi Method • Structured investigation for collecting and distilling knowledge from a group of experts through a series of questionnaires interspersed with controlled opinions and feedback (Adler & Ziglio, 1996) • Decisions on items are based on consensus of anexpert panel (Portney & Watkins, 2000)

  9. The Delphi Method • Summarized as a communications tool adhering to these principles (Linstone & Turoff, 1975): • Anonymity • Asynchronicity • Controlled Feedback • Statistical Response

  10. The Delphi Method The Delphi Method Form Expert Panel Report to Group Commenton Issue Analyze Commentary Disseminate Results

  11. Project Origination • Exert panel was organized (01/02) • Joan McMahon, facilitator & coordinator • Marj Ashcraft, data compilation & coordinator • Email solicitation to interested parties • Guidelines of participation (01/02) • Data distributed and collected electronically • Timeline of activities developed

  12. Comments on Issues • Many rich sources of information were utilized including: • Expert opinions • Practical experiences • Documented best practices identified in the professional literature

  13. Project Overview • This four round investigation collected data and integrated feedback to develop: • a set of standardized terminology related to classifications of online learning experiences • a grading rubric for assessing the quality of a course delivered online

  14. First Round • Address definitions of what is meant by “online teaching” • Expansion and deliberation of work forwarded by Judith Boettcher • Elaborate on what is meant by peer review of online instructor • Clarify and identify areas of evaluation • Six participants for this round

  15. Second Round • Clarify, expand, and modify: • Criteria for peer review evaluation • Issues related to the use of a particular criteria • Online teaching definitions • Begin development of an evaluation rubric • Four participants for this round

  16. Third Round • Integrate literature based online learning best practices into the rubric • Refine rubric, criteria and definitions • Develop a scoring form • Four participants for this round

  17. Fourth Round • Refine rubric and scoring form • Develop a pilot survey feedback instrument • Pilot the rubric and scoring form • Sent to 28 institutions for pilot (21% response rate) • Collect data • Make revisions as identified • Finalize the rubric and scoring form

  18. Web Definitions • Definitions of what it means to put a course on the Internet were clarified • Requirements • Examples • Approximate amount (%) of student time online • Differentiated between: • Web Course • Web-Centric • Web-Enhanced • Web-Presence

  19. Criteria for assessment Sixteen different criteria were developed Considerations and implications 5 point rating scale Supported with literature based best practices Rubric Criteria

  20. Navigation Course Rationale /Introduction Learning & Teaching Theories Instructional Design Goals and Objectives Learning Strategies Content Interactivity Use of Mediated Resources on the Web Assessment & Eval Internal Organization & Consistency Responsiveness to Leaner Needs Instructor’s Role Teaching Effectiveness How to Get Help Aesthetics The Rubric Criteria

  21. Tool Utilization • The developed assessment rubric can serve several purposes including: • peer review for promotion and tenure considerations • course planning • institutional policy changes • quality control issues

  22. The Tools • Peer Review for Online Learning by Ashcraft, McMahon, Lesh, & Tabrizi • http://www.towson.edu/~mcmahon/peerreview/On-linerubric.pdf • Peer Review Scoring Form by McMahon, Lesh, Tabrizi, and Ashcraft • http://www.towson.edu/~mcmahon/peerreview/PeerReviewScoringForm.pdf

  23. Grant Involvement • Quality Matters: Inter-Institutional Quality Assurance in Online Learning • FIPSE grant #P116B030646 for FY 2003 • Wells, M. (Principle Investigator). Prince George's Community College (MD) on behalf of MarylandOnline http://www.marylandonline.org

  24. References • Adler, M, and Ziglio, E, eds. (1996). Gazing into the Oracle: The Delphi Method and its Application to Social Policy and Public Health. London: Jessica Kingsley Publishers. • Boettcher, J (October,1999). Another look at the tower of WWWebble. Syllabus. p. 50-51. • Linstone A and Turoff M, eds. (1975), The Delphi Method: Techniques and Applications, Reading, MA: Addison-Wesley. • Portney, LP, and Watkins, MP (2000). Foundations of Clinical Research: Applications to Practice. Upper Saddle River, NJ: Prentice Hall.

  25. Thanks for Listening! • Happy Birthday to my youngest Daughter: Reaghan! • Q & A? • Enjoy the rest of your time here in Orlando!

More Related