An Application of the Action Research Model for Assessment Preliminary Report

1. An Application of the Action Research Model for Assessment Preliminary Report JSM, San Francisco August 5, 2003 Tracy Goodson-Espy, University of AL, Huntsville M. Leigh Lunsford, University of AL, Huntsville Ginger Holmes Rowell, Middle TN State University

2. Overview • Action Research Model • In the Context of a Collaborative Project • Showing Results from a Specific Example

3. Action Research Model* • What is the problem? I.e., what is not working in the classroom? • What technique can be used to address the learning problem? • What type of evidence can be gathered to show whether the implementation is effective? • What should be done next, based on what was learned? *1999 - R. delMas, J. Garfield, B. Chance

4. Teaching Experiment Cycle Class Implementation& Feedback Teaching Hypotheses; Curricular & Instructional Choices Instructors’ Reflections and Curricular Modifications

5. Action Research Model • What is the problem? I.e., what is not working in the classroom? • What technique can be used to address the learning problem? • What type of evidence can be gathered to show whether the implementation is effective? • What should be done next, based on what was learned?

6. Math majors who take only one course in “Probability and Statistics” are exposed to very little statistics Student understanding of complicated concepts CLT, CI’s, Combinatorics, Baye’s Theorem, ... Increase reasoning and thinking instead of memorization Better preparation for careers What is the Problem?

7. Action Research Model • What is the problem? I.e., what is not working in the classroom? • What technique can be used to address the learning problem? • What type of evidence can be gathered to show whether the implementation is effective? • What should be done next, based on what was learned?

8. What Technique can be used to Address the Learning Problem? Big Picture NSF DUE A/I Collaborative Research Award • Adaptation & Implementation of Activity & Web-Based Materials in Post-Calculus Introductory Probability & Statistics Courses • PI’s • Tracy Goodson-Espy, University of AL, Huntsville • M. Leigh Lunsford, University of AL, Huntsville • Ginger Holmes Rowell, Middle Tennessee State University *This project is partially support by the National Science Foundation. The project started in June 2002 and continues through August 2004..

9. A Collaborative Approach A&I Materials into Post Calculus Prob/Stat Courses Athens State Univ. M. Leigh Lunsford Middle Tenn. St. Univ. Ginger Holmes Rowell Univ. of Alabama, Huntsville Tracy Goodson-Espy Provide Objective Independent Assessment of A&I

10. The Materials for A&I • “A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)” • A. Rossman, B. Chance, K. Ballman • NSF DUE-9950476 • “Virtual Laboratories in Probability and Statistics (VLPS)” • K. Siegrist • NSF DUE-9652870

11. What Technique can be used to Address the Learning Problem? Specific Techniques • Increase statistical content • Change course description & course number • Understand complicated concepts and increase “thinking” instead of memorization • Change course materials used • Better preparation for careers • Integrate technology, group work, report writing

12. Action Research Model • What is the problem? I.e., what is not working in the classroom? • What technique can be used to address the learning problem? • What type of evidence can be gathered to show whether the implementation is effective? • What should be done next, based on what was learned?

13. What Evidence can be Gathered to Show Implementation is Effective? • Student’s self perception of learning of concept (survey results) • Teacher perception of student learning • In-class student feedback during the activity (continuous monitoring) • Follow-up in-class quiz • Student reports • Test/Exam questions • Student attitude survey results

14. Evidence Continued:One Example – Central Limit Theorem “Sampling Distributions of Sample Means” Computer Laboratory Simulation Activity • A. Rossman, B. Chance, K. Ballman • “A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)” • Spring 2003: used 3 of 4 examples from this activity, students wrote reports, question on test

15. Evidence (continued) CLT Example • Students’ self perception of learning • 80% (n=25) of students remembered it as an “activity that aided learning” (Fall 02 – instructor lecture & 1 SCMT example: 23%) • “My understanding of the Central Limit Theorem” • Survey question of their self reported knowledge on a scale of 1 (low knowledge) to 5 (high knowledge) • Mean response = 4.6, stdev = .64, n = 27 (Fall 02: mean = 3.8, stdev =.90, n = 25)

16. Evidence (continued) CLT Example • Teacher’s perception of student learning • In-class monitoring • Overall minimal difficulty with computer lab • Needed feedback on their understanding of the concept • Gave in-class (no credit) unannounced quiz at beginning of following class • Not enough students had gotten far enough on the activity to provide conclusive evidence

17. Evidence (continued) CLT Example • Teacher’s perception of student learning • Reports • Structure (could work in groups) • Introduction, Explanation of the 3 examples in the lab activity, Explain method used, Conclusions • Results (graded for big picture concepts and conceptual details) • Three “wrong” responses (n=27) 1) “I did not know how to do that [example 4].” • Mean gets “smaller.” • An incomplete conclusion.

18. Evidence (continued) CLT Example • Teacher’s perception of student learning • Exam questions • Students completed Example 3 (uniform lunch times) from the Course Pack activity on the test • Not previously assigned • No indication that it would be on the test • The “Uniform” subcommand was new • This portion of the test was taken in the lab • Worked individually

19. Test Question Output Variable N Mean StDev Sample means n=2 1000 29.482 12.232 Sample means n=5 1000 29.926 7.977 Sample means n=20 1000 29.826 3.801

20. Evidence (continued) CLT Example • Teacher’s perception of student learning • Results • Gave “hint” (minus 1 point) with Uniform command • Some found Uniform in the pull down menu • Biggest mistake – 1/3 of students used Uniform 30 17.32 instead of Uniform 0 60 • Filled in a table like they had done previously • Wrote a “paragraph” summarizing their findings. • Mean score 2.6 out of 3 points (st dev = 0.7)

21. Action Research Model • What is the problem? I.e., what is not working in the classroom? • What technique can be used to address the learning problem? • What type of evidence can be gathered to show whether the implementation is effective? • What should be done next, based on what was learned?

22. What should be done next, based on what was learned? • Improve my writing assignment • Improve my evaluation of student learning on this concept • Pre/post test • Use delMas/Garfield/Chance instrument (?) • Coordinate with Lunsford on this unit • Incorporate more report writing in my class • Keep using and evaluating these activity-based, discovery learning materials.

23. Resources • delMas, R., Garfield, J., and Chance B. (1999), A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students' Statistical Reasoning, Journal of Statistics Education v7, n3, http://www.amstat.org/publications/jse/secure/v7n3/delmas.cfm. • Hollins, E. R. (1999), “Becoming a Reflective Practitioner,” in Pathways to Success in School: Culturally Responsive Teaching, eds. ER Hollins and EI Oiver, Mahwah, NJ: Lawrence Erlbaum Associates. • Hopkins, D. (1993), A Teacher’s Guide to Classroom Research, Buckingham: Open University Press. • Noffke, S., and Stevenson, R. (eds.) (1995, Educational Action Research, NY: Teachers College Press.

24. Contact Information rowell@mtsu.edu