1 / 19

A progress report -- Computer coaches for introductory physics problem solving

A progress report -- Computer coaches for introductory physics problem solving. 10/30/2010 MAAPT WAPT. Qing Xu, Ken Heller, Leon Hsu, Andrew Mason University of Minnesota. Supported by NSF DUE #0230830 and DUE #0715615 and by the University of Minnesota.

erich-brady
Download Presentation

A progress report -- Computer coaches for introductory physics problem solving

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A progress report--Computer coaches for introductory physics problem solving 10/30/2010 MAAPT WAPT Qing Xu, Ken Heller, Leon Hsu, Andrew Mason University of Minnesota Supported by NSF DUE #0230830 and DUE #0715615 and by the University of Minnesota

  2. Computer coaches for Introductory Physics • Enhance Problem-Solving skills • Possible Advantages: - Provides individualized guidance and feedback outside of class; on-demand help - Effectiveness through good design and pedagogy - Customizable by instructors

  3. Research Background • Theory: - Cognitive Apprenticeship1 • Design: - Reciprocal teaching2 - Context-rich problems3

  4. 3 types of coaches • 1st: Computer coaches the student • computer decides, student implements, computer assesses • 2nd: Student coaches the computer • Student decides, computer implements, student assesses • 3rd: Student works more independently • Computer provides help as necessary; Scaffolding gradually withdraw

  5. PILOT STUDY –Fall 2010 • ~66 students, 1 lecture session of intro calculus-based physics • Assign into 3 statistically matched groups (~22 for each group) • Variables for matching: background information, e.g. HS physics & math level, FCI/CLASS/math pretests • Subset of tutors available – energy, momentum (4 weeks) • 1 Treatment group , 1 Control group and 1 placebo group • Treatment- computer coaching (on Web, outside of class), 4 problems per week • Placebo group – Work on Computer tutor problems on paper • Control group -- normal class setting • Data collection • Written solutions on quizzes & final exam • 2×4+5=13 for each student

  6. Today’s topics • Interviews – (methodological triangulation) • -- Triangulation: the use of two or more methods of data collection in the study of some aspect of human behavior. 4 • -- An important source of evidence for validity. 5 • Can written solutions well represent their Problem-Solving skills? -- Yes. • Rubric Training • Inter-rater reliability • How to score the written solutions? • -- Problem-solving Rubric. 5 • UD: Useful Description • PA: Physics Approach • SA: Specific Application • MP: Mathematical Procedure • LP: Logical Progression

  7. INTERVIEWS – Fall2010 • 5 volunteering students from introductory calculus-based mechanics course for scientists and engineers • Interviews were taken place around week 6 (Dynamics) • 1st-Kinematics/2nd-Dynamics/3rd –Kinematics • In an interview, students are asked to solve physics problems while their actions and voice are recorded (30mins). After completing the problem, they are asked to explain their reasoning to an interviewer (30mins). self-reported thought processes student written work correspond Triangulation self-reported problem-solving processes 5 rubric category processes characterize problem-solving skills from other measures of their performance Rubric scores inferences Chart1: Triangulation 5

  8. General information • 1 student actually talked out loud while he/she was working on the problem • 1 out of 5 students finished all 3 problems (1st-Kinematics/2nd-Dynamics/3rd –Kinematics) • 1 student asked for help during problem solving process • 3 out of 5 students gave up on Problem1 after the first 10 minutes; 1 student was persistent until getting a final result (15mins); 1 student got stuck and moved on the the 2nd problem(15mins) • The students didn’t use the equation sheet too much, they either saw the equations on it but remembered it anyway, or they just ignored the equation sheet.

  9. Expectations Experts Novices • Plunge directly into mathematical calculations • Do not have a very useful or complete description (picture) • Focused on the superficial features of the problem • Problem solving is a process of making a series of judicious decisions • Framework • Create useful description • Plan solution based on general principles • Carry out plan • Evaluate solution (also intermediate steps)

  10. Transcripts/interview results I: After I gave you the problem, What was the first thing that you did? S6: The first thing I did is the angle of 30°, the initial velocity. I broke it down into components. Whenever there is an angle, I would do that first, so I don’t have to worry about it later on. I: Okay, after that, what did you do? S6: Well, then I read the problem the second time. tried to absorb all the information. And, actually, I started to do my equations and backtracked and say that i don’t really know what the problem was saying. So I draw picture to kind of help me visualizing it. I kind of jumped into cracking some numbers, and then I realized, oh , wait, I don’t really get what the problem was asking, I actually go back to draw a little picture to help me visualize.

  11. interview results I: After you drew the picture, what did you do next? S6: so I went back to work on the equations, using the equation sheet. Not a bad picture

  12. Not complete

  13. Not complete

  14. Rubric Training & Scoring—Summer 2010 Training • 8 sets of previously scored student solutions • 1 undergraduate and 2 graduate students of physics • 1 experienced assessor and 2 novice assessors. • Goal:For novice assessors to achieve an increased understanding of problem solving with measurable agreement.

  15. Agreement between Raters overtime Rater 1& 2 Rater 1& 3 Figure 2: Agreement within 1 Rate 1& 36 Figure 1: Agreement within 1 Rate 1& 26

  16. Rubric Training & Scoring—Summer 2010 Scoring • 20 Students and 13 written solutions/student (4 quizzes & finals) • 1 undergraduate and 1 graduate students of physics • 2 assessors who completed training. • Goal:Establish a baseline for future stages of research • trends, dependencies( topics, gender, etc)

  17. Gender Gaps? Figure 5: Average score in each category for male and female students. Data is for the first quiz of the semester , however no statistically significant gender gap is apparent in any of the tests throughout the semester nor in student improvements from the first quiz to the final. 7

  18. Improvement Overtime? Figure 6: Changes in average rubric scores for each category from the beginning to end of the semester. Quiz 1 Problem 1 and Final Exam Problem 2 test the same physics concepts. Significant (p<0.05) gains are seen only in Physics Approach between Quiz 1 Problem 1 and Final Exam Problem 2. 7

  19. References 1. Collins, Brown, & Newman, 1989. Cognitive apprenticeship. 2. Palincsar, A.S., & Brown, A.L. (1984). Reciprocal teaching of comprehension- fostering and monitoring activities. Cognition and Instruction, 1, 117-175. 3. Heller, P., & Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. Part 2: Designing problems and structuring groups. American Journal of Physics, 60(7), 637-644. 4. Louis Cohen, Lawrence Manion, Keith Morrison, Keith R. B. Morrison, (2007). Research methods in education. 5. Docktor, 2009; Docktor & Heller, 2009 6. Talia Clark, Rubric Training Assessment Presentation, Summer 2010 7. Talia Clark, PER Final Presentation, Summer 2010

More Related