1 / 24

Computer Coaches for Problem Solving Skills in Introductory Physics: Initial Data Analysis

Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University, Winona, MN 4/30/2011. Computer Coaches for Problem Solving Skills in Introductory Physics: Initial Data Analysis.

christab
Download Presentation

Computer Coaches for Problem Solving Skills in Introductory Physics: Initial Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University, Winona, MN 4/30/2011 Computer Coaches for Problem Solving Skills in Introductory Physics: Initial Data Analysis Supported by NSF DUE #0230830 and DUE #0715615 and by the University of Minnesota

  2. Outline • Background • Demonstration • Preliminary findings • Initial calibration data • Currently: Scoring using Rubric for Problem Solving (Docktor 2009) • Methodology for further study

  3. Research Study - Basic Method • Use coaches in a calculus-based physics course for scientists and engineers • Students are asked to volunteer; volunteers are paid for their participation • Assign students into 2 matched groups • Variables for matching: background information, e.g. HS physics & math level, FCI/CLASS/math pretests

  4. Previous Study: Overview • Study #1 (Fall 2010) • ~40 students, 1 lecture session of intro calculus-based class (~20 for each group) • Subset of coaches available – energy (8), momentum (7) – done over 4 weeks (~4 per week)

  5. Preliminary Results, Fall 2010 • Retention rate: high in fall (18 of 21) for 15 coaches • 2 others completed 12 of 15 • All students found them useful • Does this hold for other sections? • What will happen with a larger set of coaches?

  6. Preliminary Results, Fall 2010 • Student preference for each type • Faculty tend to disagree with students (found type 1 tedious) • 1 student initially preferred type 1 but switched to type 3 after gaining familiarity with physics

  7. Time between questions • Average time range to complete between coaches: between 20 and 40 minutes • Percentage of correct answers seems to correlate with background info • 2 sampled “A” students: 80-90% • 2 sampled “C” students: 60-70% • Students tend to stay on task • Only 2 of 18 had at least one break of more than 5 minutes for a question

  8. Sample of logging function data • Time between questions(one type 1 coach) • Distribution suggests students are taking the tutors seriously • Median time = 4.58 s

  9. Evaluating Problem Solving(Docktor 2009; Docktor and Heller 2009) • Rubric developed to evaluate student problem solutions • Validity, reliability have been tested • 2 raters use, discuss with each other until >90% agreement • Five rubric categories (established by research on expert and novice problem solvers) • Useful Description • Physics Approach • Specific Application of Physics • Mathematical Procedures • Logical Progression

  10. Current Study: Establishing a Baseline • Study #2 (Spring 2011) • 9 students, 1 lecture session of intro calculus-based class • Coaches available for 4 segments: kinematics (3), dynamics (4), COE (8), COM (7) • Good time to establish baseline of rubric scores for general class • Eventual comparison with computer coach users • 2 expert raters – PER researchers • 23 students (3 tiers according to pretests); eventually will expand to ~40 • 13 problems (8 from quizzes, 5 from final)

  11. Sample Problem from Q2

  12. Sample Solution Rating(Before Discussion)

  13. Baseline rubric scores - Standard error bars are on the order of +/- 1 to +/- 2 out of 5

  14. Baseline rubric scoring patterns • Raters: 2 experts/PER researchers

  15. Baseline rubric scoring patterns

  16. Plan for larger-scale study • ~90 students, 1 lecture session of intro calculus-based class (~45 for each group) • Comparison group given equal face time with problems used in coaches • All coaches available (kinematics, dynamics, energy, momentum, rotational motion) • 8 x 5 = 40 total

  17. Questions to Be Addressed • Short-term questions: • Will students use coaches? • How will students use them? (keystroke function) • Do they improve students’ problem solving skills with respect to baseline scores? (rubric scoring of quizzes)

  18. Questions to Be Addressed • Longer-term questions: • Are they adaptable to be used in teaching other physics courses? • Possible software/AI development – refine beta versions; make it more able to follow student preferences • Can this software be modified by instructors to fit a different problem solving framework?

  19. Thanks! • Summary • Initial results seem to have much to say; need to be expounded upon • Currently examining a baseline of exam performance to compare to future data from computer coach users • Website: http://groups.physics.umn.edu/physed • Can look at research, previous talks and publications • Try out the coaches! Give us feedback! http://groups.physics.umn.edu/physed/prototypes.html

  20. Basic Method • Treatment and Comparison groups • Treatment group – computer coaching (on Web, outside of class), 4 problems per week • Comparison group – normal class setting • Data collection • Diagnostic pretests and posttests • Written solutions on quizzes & final exam • 2×4+5=13 for each student • Problem-solving interviews with students

  21. Individual student data • Can look at individual time on each question for each student • A few questions take some time regardless of student • Entering answer into calculator • Can look for patterns in other questions

  22. References • Chi, Feltovich and Glaser, “Categorization and representation of physics problems by experts and novices,” 1981. • Collins, Brown and Newman, “Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics,” 1989. • Docktor and K. Heller, “Robust assessment instrument for student problem solving,” 2009; also see J. Docktor, dissertation, 2009. • P. Heller and K. Heller, “The Competent Problem Solver: A Strategy for Solving Problems in Physics,” 1995. • P. Heller and Hollabaugh, “Teaching Problem Solving Through Cooperative Grouping. Part 2: Designing Problems and Structuring Groups,” 1992. • Hsu, Heller, Mason, and Xu, Summer AAPT presentation, Portland, OR, 2010.

  23. References • Larkin, McDermott, D. Simon and H. Simon, “Expert and Novice Performance in Solving Physics Problems,” 1980. • Newell and Simon, “Human Problem Solving,” 1972. • Palincsar and Brown, “Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities,” 1984. • Polya, “How to Solve It,” 1945; 1957. • Polya, “Mathematical Discovery,” 1962. • Reif and Scott, “Teaching Scientific Thinking Skills: Students and Computers Coaching Each Other,” 1999; also see L. Scott dissertation, 2001.

More Related