More Writing But Less Grading: Calibrated Peer Review™

1. More Writing But Less Grading: Calibrated Peer Review™ Share The Future IV - Tempe, AZ John Wise The Pennsylvania State University

2. Penn State’s Engineering Instructional Services • College of Engineering, University Park PA • Associate Dean for Undergraduate Studies, Robert Pangborn • Support and Training for Faculty and Teaching Assistants Engineering Instructional Services eis@engr.psu.edu

3. Review of Bloom’s Taxonomy • Knowledge • Comprehension • Application • Analysis • Synthesis • Evaluation Bloom (1956) Taxonomy of educational objectives: the classification of educational goals Engineering Instructional Services eis@engr.psu.edu

4. Problem Statement • Large Classes in Particular: • Predominantly Lecture • Multiple Choice / “Scantron” Grading • Feedback and Writing Difficult • All Classes • Student “Final” Papers Often First Drafts • Product vs. Process Orientation • Concept Misunderstandings Engineering Instructional Services eis@engr.psu.edu

5. History of Calibrated Peer Review (CPR) • Large Molecular Science Sections @ UCLA • Science-Based Model • Research, Write, Peer Review • Now Can Serve Any Discipline and Educational Level Engineering Instructional Services eis@engr.psu.edu

6. How It Works • Create (or Select) Assignment • Set Timing and Standards • Manage Assignments • Review Results Engineering Instructional Services eis@engr.psu.edu

7. Create Assignment • Choose Source • Define Writing Assignment • Write Exemplar • Write Moderate / Poor Examples • Generate Style and Content Questions • Assess Own Documents for Calibration (sample) Engineering Instructional Services eis@engr.psu.edu

8. Set Timing and Standards • Control Over Text Entry Times • Control Over Calibration Standards • Control Over Scoring Scheme Engineering Instructional Services eis@engr.psu.edu

9. Scoring Template: High Difficulty Calibrations To master each calibration students must: • Answer3.0of  4 (75%) STYLE questions correctly, and • Answer 5.3of 7 (75%) CONTENT questions correctly, and • NOT deviate by more than 2.0 points from the RATING of a calibration text Reviews To master reviews students must: • NOT deviate by more than 2.0 points from the AVERAGE RATING of the reviewed text Self-Assessment To master self-assessment students must: • NOT deviate by more than 1.0 points from the AVERAGE RATING of their text to receive full credit, or • NOT deviate by more than 2.0 points from the AVERAGE RATING of their text to receive half credit

10. Directions: Enter the points you want associated with each assignment stage. All points must sum to 100. Each point total can range from 0 to 100. 1.Text quality: XX points 2. Calibrations: XX points over 3 Calibrations 3. Reviews:  XX points over 3 Reviews 4. Self-assessment: XX points TOTAL: must sum to 100 points

11. Management • Student Reminders • Special Requests • Late Submissions • Technical Difficulties Engineering Instructional Services eis@engr.psu.edu

12. Review Results • Scan Student Results • Identify “Problem” Scores • Ability to Randomly Review Some or All Student Submissions • Can also participate AS a student! Engineering Instructional Services eis@engr.psu.edu

13. Engineering Instructional Services eis@engr.psu.edu

14. Student Point of View • Review Materials • Write • Calibrate • Review three examples, compared to professor • Review 3 Random Peers • Review Self Engineering Instructional Services eis@engr.psu.edu

15. Calibrated Peer Review ™ (CPR) • Available from UCLA • http://cpr.molsci.ucla.edu • Contact me for sample assignment • jwise@engr.psu.edu • 814-865-4020 • Contact UCLA regarding traveling workshops Engineering Instructional Services eis@engr.psu.edu

16. More Writing But Less Grading: Calibrated Peer Review™ Share The Future IV - Tempe, AZ John Wise The Pennsylvania State University