Learning to assess motor skills via an individualized web-based application

1. AAHPERD 2009 Tampa, Florida Learning to assess motor skills via an individualized web-based application Luke E. Kelly, Ph.D., CAPE University of Virginia Tom Moran, CAPE James Madison University

2. Presentation Outline • The Need • The Challenge • A Possible Solution • Questions and Discussion

3. Poll Audience • What level do you currently teach at? • Elementary • Middle • High School • College/University • If you taught a lesson for 100 minutes, how many minutes would you spend on assessing? • Write the first three things that come to your mind when you hear the term “Assessment.”

4. Assessment Review • Teaching Level • Application designed for both • Practicing teachers • University teacher trainers • Time spent on assessing • Assessing viewed as an integrated part of teaching • You need to continuously assess in order to teach • Feelings regarding assessment • Love it, fun, easy, rewarding, enjoyment, self-fulfilling, the key to my teaching, …… • Hate it, time consuming, waste of time, impossible to do in my setting, barely have enough time to teach …

5. Clarify What We Are Talking About? • Part of the problem is terminology • When I say “assessment” • I am thinking curriculum embedded, on-going, formative, qualitative, criterion-referenced, observations that are an integrated part of the teaching process • You may hear a mandated, time consuming process, that does not help your instruction, is hated by the students, and that wastes lots of your limited instructional time and primarily focuses on only one component of physical education – fitness.

6. So what is Assessment? • Assessment is the process used by teachers to make informed decisions about what to teach, when to teach it, and how to teach it so that it is learned by ALL students. • Assessment is also the process by which students learn – informs them what components of a skill they can and cannot currently perform.

7. Terminology General Specific • Assessing • Testing • Evaluating • Accountability • IDEA • NCLB • SOLs • Norm-referenced • Standardized • Quantitative • Product oriented • Summative • Criterion-referenced • Qualitative • Process oriented • Formative

8. The Basics • How are motor skills traditionally taught without assessment? • Throwing example: • Demonstrate the throw • Emphasize why this skill is important • Engage the class in an active activity involving throwing • Move around the class and give feedback as needed • Conclude with a group activity that uses throwing • Encourage the students to practice outside of class

9. Student’s View of the Demonstration • What do I need to do to learn this skill? • Observe the teacher’s demonstration • Task analyze the skill I observe down into its key components • Self assess myself to determine which of these components I can already perform and which ones I still need to learn • Prioritize the components I need to learn, so I know what to concentrate on first • Figure out how to fix my performance on that component • Practice the new behavior during the class activities • Show the teacher when I think I have it

10. Collaboration • How can we work together with our students to maximize their learning? • Teach them the components • Assess them and let them know: • What components they can already do • What components still need some work • Which component(s) they should focus on • Teach them how they can self-assess themselves • Plan and implement instructional activities that: • Focus on specific components of the skill • Provide immediate feedback

11. The Weak Link • The problem is not: • Planning instruction • Creating fun and challenging activities • Class management • The problem is assessing competency • Teachers are not being trained to competency on how to qualitatively assess • Teachers subsequently are not successful at qualitative assessment when they teach, so that do not do it. • Ability to qualitatively assess motor skills is an essential skill for physical educators (Barret, 1979; Brown, 1982; Hoffman, 1977; Horvat, Block, & Kelly, 2007; Kelly & Melograno, 2004); Lewko, 1976; Lounsbery & Coker, 2008; Thomas & Thomas, 1983; Werder & Kalakina, 1985; Wessel & Kelly, 1986) • Physical educators are not competent in qualitative assessment (Behets, 1996; Gallo, 1999; Lounsbery & Coker, 2008; Stroot & Oslin, 1993;Veal 1992; Walkley & Kelly, 1989)

12. The Problem is Well Documented • Ability to qualitatively assess motor skills is an essential skill for physical educators need (Barret, 1979; Brown, 1982; Hoffman, 1977; Horvat, Block, & Kelly, 2007; Kelly & Melograno, 2004); Lewko, 1976; Lounsbery & Coker, 2008; Thomas & Thomas, 1983; Werder & Kalakina, 1985; Wessel & Kelly, 1986) • Physical educators are not competent in qualitative assessment (Behets, 1996; Gallo, 1999; Lounsbery & Coker, 2008; Stroot & Oslin, 1993;Veal 1992; Walkley & Kelly, 1989)

13. Why is assessing a problem? • We teach hundreds of skills • You can not learn to qualitatively assess motor skills by yourself • You need immediate and accurate feedback on the accuracy of your judgments • You were taught about assessment, but in most cases, not trained to competency in doing it • If you are not good at something and can avoid doing it, you cope and avoid

14. How can this problem be addressed? • Harness the power of computer technology • CAI has been shown to be an effective way to train teachers in assessment (Kelly & Moran, in press; Kerns, 1989; Kulik & Kulik, 1991; McKethan & Evhart, 2001; Walkley & Kelly, 1989; Williams & Tannehill, 1999) • Major problem in the past has been equipment cost and compatibility • Web-based/internet applications solve these problems

15. MSAPMotor Skill Assessment Program • Currently a skill specific research tool • Effectiveness compared to traditional methods • MSAP more effective • Validity • Does training generalize to live assessing? • Effectiveness with preservice and inservice teachers • Preservice students do not act like adult learners • Goal is create a public access application • Internet accessible 24/7 • Include full range of skills

16. Overview of MSAP Features • Tutorial • Learn and internalize the focal points by seeing correct and incorrect examples • Create a mental image of the correct performance • Guided Practice • Practice assessing students with control over the number of trials and the speed of performance • Receive immediate focal point specific feedback on your performance • Competency Evaluation • Designed to simulate actual assessment conditions • 10 clips, shown only 3 times, only in real speed • Only receive summative feedback

17. MSAP DEMONSTRATION (Start)

18. MSAP Development Process • Select and assessment item • MSAP uses Everyone CAN assessment items • Create a pool of 150 video clips • Make a list of specific errors and correct performances needed • Do not just hope if you film enough kids you will get all the clips you need • Film students • Prompt/guide some students to get needed performances • Convert clips to Flv computer files

19. Development (cont) • Distribution of clips • Tutorial – 30 clips • Correct performance of each focal point • Common errors on each focal point • Guided Practice – 90 clips • Program randomly selects 5 clips for each practice trial • Competency Assessment – 30 clips • Presented in sets of 10 with only summative feedback • Create answer key for all clips • Have answers validated by external experts • Create error explanations for each clip • Enter clip data into MSAP and test

20. Current MSAP Research • Ongoing research • Validity of MSAP with preservice majors and inservice teachers • Effectiveness with inservice physical education teachers • Simulated vs. real time assessment? • How long does it take teachers to be trained to a level of competency? • Assessing competency retention • Continue to expand & improve MSAP • Kicking • Underhand roll • Catch

21. Summary • To teach without assessing is comparable to a carpenter building a house without a tape measure. You can work at it, probably harder than you should have to, and you will produce something, but not the best product you could have. • Curriculum embedded, on-going, qualitative assessment is the key to both effective teaching and student learning. • Only you can change your teaching behavior.

22. Contact Information • Luke E. Kelly • University of Virginia • E-mail: lek@Virginia.edu • Phone: 434-924-6194 • Tom Moran • James Madison Univ. • E-mail: morante@jmu.edu • Phone: 540-568-4877

23. QUESTIONS DISCUSSION