Measuring Physical Activity for Program Evaluation

1. Measuring Physical Activity for Program Evaluation Smith Child Health Research Outreach and Advocacy Center Evaluation Core September 10, 2019

2. Plan for today • Introductions • Choosing a measurement option • SOPLAY and SOFIT • Talk about uses • What it can do • Go through protocol • Practice • Analysis • Pedometers • Accelerometers • Discussion/questions

3. Choosing a method

4. SOPLAY: System for Observing Play and Leisure Activity in Youth

5. What data does it give you? • Child physical activity levels • Minutes and % of time spent in sedentary, walking, or vigorous activity • Optional measures: • Condition of observation area • Most prominent type of physical activity that children are participating in

6. What can you do with that info? • Gather an objective assessment of child activity during free play or unstructured play time in your programs. • Recess, lunch time, summer camp, etc. • Compare activity level of boys and girls for different programming activities. • Compare between sites with same programming. • Pre-post comparison before and after a change at a single site. • Calculate an average of activity levels in your program to report to funders. • Low-cost and relatively low-burden.

7. Protocol highlights • Decide on scanning interval frequency • Decide how many minutes to spend on each scan space, then rotate to next scan space • Decide on scanning areas • Okay to split up target area into smaller sections (“scan spaces”) • Keep scan spaces consistent for all observations (i.e. create a map for future reference, if conducting observations at multiple time points)

8. Map examples

9. Protocol Highlights • Keep order and duration length of rotations consistent for all observations • Scan boys and girls separately. • Always scan from LEFT to RIGHT. Observe each student in the area once.

10. Protocol Highlights • Write down the number of boys and girls observed in sedentary (S), walking (W), or vigorous (V) activity during the scan. • Tip when observing large groups: Keep a separate sheet to tally # of boys and girls observed • Do not back-track to count new children entering the scan area. • Good to have two people to confirm assessment. • Good to have same person if comparing pre-post.

12. Analysis step by step • To calculate the numerator add down the column for each activity level • This is the total “minutes” a person was in this state of activity • These numbers will likely all be different

13. Analysis step by step • To calculate the denominator add all columns for each sex • This is the total “person minutes” • Will be the same if you have a constant number of participants • Double Check • Sum each row and add together • Sum total activity minutes

14. Analysis step by step • To calculate the percent of time spent in an activity state divide the Total Activity Minutes by the Total Person Minutes • 37/100 = 37% • 38/100 = 38% • Etc.

15. Practice!!

16. SOPLAY practice • Split into two groups • Group 1 will observe; Group 2 will play a game • -Swap- • Group 1 will play a game; Group 2 will observe • Discussion

17. SOFIT: System for Observing Fitness Instruction Time

18. What data does it give you? • Student physical activity levels • Minutes and % of lesson time spent in MVPA • Lesson context • Minutes and % of lesson time spent doing different activities • Instructor behavior • Percent of lesson intervals spent promoting activity and fitness during and out of class time

19. What can you do with that info? • Gather an objective assessment of child activity in your fitness/PE programs. • Gather objective measures of specific children’s activity. • Compare activity level of boys and girls for different programming activities. • Compare between sites with same programming but different staff/implementation. • Pre-post comparison before and after a change at a single site. • Calculate an average of activity levels in your fitness/PE program to report to funders. • Low-cost and relatively low-burden.

20. Protocol highlights • Identify the questions you want to answer: • What percent of PE class time are students engaged in MVPA with our current PE curriculum? • Which activities/lessons create the most MVPA in our students? • Did student A engage in more minutes of MVPA in PE at the end of the school year than in the beginning?*Not designed for this but can do it* • Decide: • Measurement period • Three days of classes, one full week, etc. • Number of PE classes/grades to be observed • Number of observations per school/grade/PE class during the measurement period • Schedule of observations

21. Protocol highlights • If you are assessing PE overall you want to choose lessons that are representative of the school: • Different grade levels • Lesson content (e.g. – a specific unit on basketball) • Days of the week • Different PE teachers • If you are looking at one grade or class or lesson type then you can stick to that – but then you can’t generalize findings to the school as a whole.

22. Protocol highlights • Factors to keep in mind when selecting lessons: • Instructional goals – fitness, skill, knowledge, social/emotional development • Instructional content – type of unit, lesson placement in the unit • Class characteristics – size, diversity • Environmental conditions – indoor/outdoor, class size, weather

23. Protocol highlights • SOFIT captures information on 3 categories at the same time: • Student physical activity • Lesson context/content • Teacher behavior • Information is observed during a 10 second observation window, then recorded on the data sheet during another 10 second window. • Pens, data sheets, pacing tape and something to write on.

24. SOFIT protocol • Select 5 students who will be the representatives for the class • For class size <25 select the 4th, 8th, 12th, 16th and 20th students to arrive • For class size >25 select the 5th, 10th, 15th, 20th, and 25th students to arrive • Okay to write down characteristics of student to help jog your memory (e.g.- brown glasses, red shirt, Nike shoes). • Observe each student for 4 consecutive minutes before changing to the next student. • Reserve the 5th student as a backup in case one of your 4 primary students leaves.

25. SOFIT protocol • Observing activity level through body position • Lying down = 1 • Sitting = 2 • Standing = 3 • Walking = 4 • Vigorous = 5

26. SOFIT protocol • Vigorous (5) is anything that takes more energy than regular walking • standing up from laying down • sit-ups/push-ups • running • riding a stationary bike • wrestling • If a student moves from one activity level to another during the 10 seconds, code for the last activity level. Technically you only code for the activity happening at the “record” prompt at the end of the observe period. • If the student is transitioning at that exact “record” moment code-up the higher level of activity.

27. SOFIT protocol • Observe what is being done in the class – the “activity” itself • General content = M • Knowledge content = K • Fitness activity = F • Skills practice = S • Game play = G • Other = O

28. SOFIT protocol • General content (M) • Lesson time when students are not intended to be active • Transition time between activities • Breaks • Class management • Organizing activities • Attendance • Changing rooms • Team selection • Getting a drink of water • Collecting money for field trip • Social chatting/hanging out

29. SOFIT protocol • Knowledge content (K) • Lesson time when the focus is on student acquisition of knowledge related to PE. • PA concepts • PA/game history • Health benefits of exercise • Reasons for stretching/warm-up/cool down etc. • Technique • Rules of a game

30. SOFIT protocol • Fitness (F) • Time doing activities whose major purpose is to alter the physical state of the individual. • Calisthenics • Long distance running • Weight training • Agility training • Fitness testing • Warm-up • Cool down • Cardio endurance

31. SOFIT protocol • Skill practice (S) • Activity time allocated to practice of skills with goal of skill development • Volleyball drills • Hurdle drills • Balance skills • Dance steps • Swimming stroke practice

32. SOFIT protocol • Game play (G) • Activity time devoted to the application of skill in a game or competitive setting. • Sport games (volleyball, basketball soccer etc.) • Dance performances • Games of tag • Gymnastics routines

33. SOFIT protocol • Other/Free play (O) • Free time during which physical education instruction is not intended.

34. SOFIT protocol • Teacher promotion of physical activity • When in-class PA is promoted code = I • Good job, keep it up • 20 more sit ups guys • Good hustle • When out-of-class PA is promoted code = O • I want everyone to practice tonight when they get home • Don’t forget to sign up for the track team • Pat, great job at the soccer game last weekend • When no PA is being promoted code = N • This is the default code

35. SOFIT protocol • The interaction can be directed to any student in class, not just the target student. • You may code for both I and O • Alternate measure of teacher involvement • Promotes fitness • Demonstrates fitness • Instructs generally • Manages • Observes • T – other tasks

36. SOFIT protocol • Place yourself in a location where you can see your target student and hear the teacher. Be prepared to move around. • Do not begin observations until teacher is present. • Start the pacing tape and begin observations when 51% of the students reach the instructional station (i.e. gym, outdoor field, etc.) –write down the time. • Only observe a class/lesson if you will be able to capture the whole lesson period (do not start more than 5 minutes late or end before 51% of the students have left).

37. SOFIT protocol • Observe the student, context and teacher for 10 seconds and use the next 10 seconds to enter the information in the data sheet • Code student 1 for 4 consecutive minutes then move to student 2, 3 and 4 in sequence. Continue in this manner until the lesson ends. • If one of the students leaves the class during an observation, switch to the alternate. Continue with the alternate even if the original student returns. • If a student leaves the class but returns in time for their observations continue with that target student.

38. SOFIT analysis • Calculate and record the lesson length on page 1 of your data sheets • Add up the total (vertically) for each coding category at the bottom of each page • Copy the summary scores from each page to the SOFIT Summary Form • Calculate the total (across all pages) and record under total • Divide each total by 3 to convert into minutes • Divide new totals by the total lesson minutes for % of lesson time

39. Practice!!

40. Lesson Context: https://www.youtube.com/watch?v=cAWykAFU6Vk • Everything • https://www.youtube.com/watch?v=DTsKnTNTB_k

41. Questions?

42. Resources • CLOCC Accelerometer library • SOPLAY counters • SOPLAY analysis sheet • SOFIT analysis sheet • Contact Sarah Welch, swelch@luriechildrens.org, 312-227-7045

43. Resources • SOPLAY Protocols • https://activelivingresearch.org/soplay-system-observing-play-and-leisure-activity-youth • SOFIT Protocol • https://activelivingresearch.org/sofit-system-observing-fitness-instruction-time • SOFIT training videos • Examples of codes and categories • https://www.youtube.com/watch?v=HiYTB_ee3t0 • Practice • https://www.youtube.com/watch?v=DTsKnTNTB_k