Using Technology for Content Delivery, Formative Assessment, and Reflection

1. Using Technology for Content Delivery, Formative Assessment, and ReflectionMichael S. KirkpatrickJMU Computer ScienceBridgewater CollegeAnnual Pedagogy Project 2015 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

2. Workshop Objectives • At the completion of this session, participants will demonstrate progress toward the following objectives: • Summarize relevant literature for active learning • Explain how to use videos for effective content delivery • Describe how to incorporate formative assessment into a flipped classroom • Identify practical strategies to encourage students’ day-to-day class preparation and metacognition Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

3. Agenda • Introductions and welcome • Active learning preassessment • Camtasia lessons learned • Formative assessment and PI • Metacognition and reflection • Discussion and exploration Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

4. “Adopting instructional practices that engage students in the learning process is the defining feature of active learning.” -Michael Prince Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

5. Benefits Illustrated • Measure of performance gain • Mechanics Diagnostic (MD) or Force Concept Inventory (FCI) • 62 courses (14 trad.) at multiple institutions • 6542 students (2084) R.R. Hake, "Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66, 64- 74 (1998). http://www.physics.indiana.edu/~sdi/ajpv3i.pdf Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

6. Collaborative and Cooperative Learning M. Prince, “Does active learning work? A review of the research.” J. Eng. Education 93(3), 223- 241, 2004. http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Prince_AL.pdf Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

7. Problem-Based Learning M. Prince, “Does active learning work? A review of the research.” J. Eng. Education 93(3), 223- 241, 2004. http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Prince_AL.pdf Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

8. Active Learning in STEM • Meta-analysis of 225 studies • 158 studies: average 0.47 SDs better on CIs/exams • 67 studies: average failure rate dropped from 33.8% to 21.8% with active learning S. Freeman et al., “Active learning increases student performance in science, engineering, and mathematics,” Proceedings of the National Academy of Sciences 111(23), 8410- 8415, 2014. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4060654/pdf/pnas.201319030.pdf Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

9. From Content Delivery to Formative Assessment Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

10. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

11. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

12. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

13. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

14. Lessons Learned • Be mindful of accessibility • Never use red and green contrasts • Create subtitles or transcripts as needed • Write a script first • Doing so will save you time later • Makes for easy transcript creation • Align videos with learning objectives • Use a discussion forum for students to ask questions • Incorporate an ungraded Moodle quiz for practice Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

15. Formative Assessment and Peer Instruction Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

16. “Feedback is value-neutral help on worthy tasks. It describes what the learner did and did not do in relation to her goals. It is actionable information, and it empowers the student to make intelligent adjustments when she applies it to her next attempt to perform.” -Grant Wiggins Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

17. Peer Instruction • Peer Instruction • Created by Eric Mazur (Harvard) • Augment class with ConcepTests • Expose common misconceptions • Think-vote-pair-revote pattern • E. Mazur, Peer Instruction: A User’s Manual, 1996.http://mazur.harvard.edu/research/detailspage.php?rowid=8 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

18. Physics Question 1 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

19. Closing the Gender Gap • Cooperative learning closes the gender gap • Pretest scores were 10% points higher for men • Gap persisted with lecture alone • Posttest results for cooperative classes were almost equal • PI can eliminate gender gap in physics • T: traditional lectures • IE: interactive lectures • IE+: interactive assignments, lectures, tutorials E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary talk at the International Computing Education Research Conference (ICER), 2011. http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

20. Closing the Gender Gap • Traditional lectures leave women behind • Women tend to have smaller performance gains • Cooperative learning improves gains for women • ...but men improve as well E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary talk at the International Computing Education Research Conference (ICER), 2011. http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

21. Physics Question 2 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

22. Demos and Engagement • Performance and understanding increase with engagement • Those who only observe sometimes learn it wrong • Those who discuss show clearer reasoning and provide partially correct answers Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

23. Physics Question 3 Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

24. Confusion and Understanding • “Please tell us briefly what points of reading you found most difficult or confusing.” • “Nothing was difficult or confusing.” • “I found the explanation inadequate. I don’t understand the reasoning that led to the conclusion.” Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

25. Best Practices for Formative Assessment • Identify the learning gap • Space between what students know and need to know • Bidirectional feedback • Identify student progress and suggest corrections • Actively engage students • Students need to assess their own understanding • Create learning progressions • Break larger goal into subgoals Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

26. Best Practices for Formative Assessment • Ask clear questions • Avoid ambiguity, use only one verb • Psychological safety • Positive reinforcement, accept imperfection • Sequencing and balance • Consider the order and type of questions, activities • Wait time • Come ot terms with silence • Avoid pimping questions • Do not try to establish intellectual superiority Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

27. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

28. Metacognition Space Race Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

29. Metacognitive Activities • Preassessments • What do I already know about this topic? • Muddiest points • What am I still confused on? • Exam corrections • Why did I miss this question? • Documented problem solving • What were the steps I used to solve this problem? • Learning progress journals • How did my understanding of this concept change? Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

30. RSQC2 • Recall the most important points • Requires student to analyze relative importance • Summarize the most important points • Provides practice with comprehension • Construct a question you would like answered • Encourages reflection and evaluation • Connect this material to other concepts • Establishes scaffolding from previously learned material • Comment on your learning progress this week • Uses metacognitive reflection to instill study habits Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

31. Tools for Metacognitive Activities • Socrative • Good use: quick in-class polling • Qualtrics • Good use: offline, anonymous surveys • Moodle • Good use: graded and offline assessments • Social media • Good use: blogs as learning journals • Piazza • Good use: collaborative student discussions and answers Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

32. Best Practices for Metacognition • Develop metacognitive culture • Give students freedom to be confused • Integrate reflection into credited course work • Model metacognitive behavior • Teach the concept and language of metacognition • Explicit instruction over time expands skill set • Reflect the specific learning context • Metacognition is NOT generic • Externalize mental events • Increase accurate awareness of strengths and weaknesses Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

33. Bloom’s Taxonomy Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

34. Discussion and Exploration Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

35. Other Technology Tools • Twitter • Good use: announcements, highlight news stories • CATME • Good use: team formation, peer evaluation • Top Hat Monocle • Alternative to Socrative (requires paid subscription) • Asynchronous MOOC Videos • edX, Kahn Academy, MIT OpenCourseWare Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

36. Resources • Peer Instruction (PI) • E. Mazur, Peer Instruction: A User’s Manual, 1996. • http://mazur.harvard.edu/research/detailspage.php?rowid=8 • Formative Assessment • T. A. Angelo and K. P. Cross, Classroom Assessment Techniques: A Handbook for College Teachers, 1993. • http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessment.html http://cft.vanderbilt.edu/guides-sub-pages/cats/ • Metacognition • E. Cook, E. Kennedy, S. Y. McGuire, “Effect of Teaching Metacognitive Learning Strategies on Performance in General Chemistry Courses,” J. Chem. Educ., 2013, 90 (8), pp 961–967. • http://www.lmu.edu/Assets/Centers+$!2b+Institutes/Center+for+Teaching+Excellence/Teach+STEM+Students+How+to+Learn--Metacognition+is+the+Key!+Slides.pdf Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick

37. Resources • Others • J. D. Bransford, A. L. Brown, and R. R. Cocking, How People Learn: Brain, Mind, Experience, and School, 2000. • M. Weimer, Learner-Centered Teaching: Five Key Changes to Practice, 2002. • G. Wiggins and J. McTighe, Understanding by Design, 2005. • N. Pinchok and W. C. Brandt, “Connecting Formative Assessment Research to Practice: An Introductory Guide for Educators,” 2009. http://www.learningpt.org/pdfs/FormativeAssessment.pdf • N. Chick, Metacognition. http://cft.vanderbilt.edu/guides-sub-pages/metacognition/ • T. Tofade, J. Elsner, S. T. Haines, “Best Practice Strategies for Effective Use of Questions as a Teaching Tool,” Am J Pharm Educ. 2013 Sep 12; 77(7): 155. Using Technology for Content Delivery, Formative Assessment, and Reflection Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick