1 / 14

Developing evaluation in physical activity projects: project workers’ views and actions

Developing evaluation in physical activity projects: project workers’ views and actions. Adam Lockwood Faculty of Health and Social Care The Open University A.lockwood@open.ac.uk. PhD research.

akiva
Download Presentation

Developing evaluation in physical activity projects: project workers’ views and actions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Developing evaluation in physical activity projects: project workers’ views and actions Adam Lockwood Faculty of Health and Social Care The Open University A.lockwood@open.ac.uk

  2. PhD research • Focused on researching evaluation in chances4change projects that aim to encourage children to take part in more physical activity • Background • My literature review revealed that little is known about the evaluation of such projects (Stratton et al. 2005; Van Sluijs, 2007) • However it is clear that there may be multiple barriers affecting the execution of evaluation and its utilisation in community-based projects.

  3. PhD research • Such barriers may reflect: • resource constraints (Evaluation tools)(Dugdill and Stratton, 2005) • but they may also relate to attitudes and values of project staff (e.g. Whether they see it as a worthwhile task)(Seppänen-Järvelä, 2004; Ruch Ross et al. 2008; Taut and Alkin, 2003)

  4. Overview of the PhD project • Explore how evaluation is understood and handled in chances4change physical activity projects that work with children • Semi-structured interviews with project managers/leaders and workers and a Desk-based review of chances4change evaluation documentation • Using this research in triangulation with my literature review, I will design an evaluation toolkit • Pilot study: Test out the evaluation toolkit. Using non-participant observation and semi-structured interviews I will look at how the toolkit is used and look at the project workers views of evaluation have changed

  5. Overview continued • After modifying the toolkit and research methods I hope to use the toolkit more widely in six chances4change projects and/or partnership organisations (Also using observations and interviews) • All of the findings from this research will feed into my PhD that examines the meanings, uses and potential of evaluation

  6. Progress with the fieldwork • Desk-based review looking at evaluation documents from ten chances4change projects • Six semi-structured interviews • One with the Programme Manager and Research and Evaluation Manager for chances4change • Two project workers • Two project leads

  7. Findings from the initial interviews and desk-based review Barriers to evaluation • It is difficult to evaluate over the whole age range (5-16). Particularly difficult to evaluate 5-8yrs (Literacy issues) • It is a challenge for children to recall information on their levels of physical activity • Children struggle to understand the evaluation questions that are being asked or they cannot express themselves easily • Children feel that they are being graded during the evaluation. ‘Am I answering the question correctly?’

  8. Barriers to evaluation continued • The type of projects: one off sessions vs long term projects (struggle to evaluate one off sessions) • Timing of the activities (not enough time to conduct the evaluation, pressures with project delivery) • Project workers think that evaluation involves number crunching (This often puts them off) • Some projects do not know how to analyse and interpret the evaluation data or the evaluation in itself doesn’t have a great deal of meaning

  9. Findings from the initial interviews and desk-based review Training • There are different levels of experience amongst project workers (The toolkit should include evaluation training ) • The training needs to be quick and simple so that the project workers don’t lose interest or become overwhelmed • ‘The evaluation needs to be given meaning’, explain why and how projects can evaluate their work • ‘The data takes hours to analyse’, the toolkit should include a section on how to analyse evaluation data

  10. Findings from the initial interviews and desk-based review Make the evaluation fun for children • Toolkit that is engaging- e.g. ‘If the toolkit includes a self report measure such as a questionnaire let the questions include smiley faces and pictures and make it colourful’ • Integrate the evaluation activity into the typical project activities (e.g. Include a fun active game) • Questionnaires are very boring for children, include gadgets or something interactive like a game • Evaluation can be interactive (answer questions by throwing balls into buckets and by putting stickers on charts)

  11. Findings from the initial interviews and desk-based review Current approaches used in projects: • Questionnaires, focus groups, and physiological measurements (BMI, Multistage running tests) Ideas for the toolkit: • Visual Likert scales (Using stickers, throwing balls into buckets) • Paired interviews , focus groups, using photographs as visual prompts , big brother diary room • Postcard evaluations (fun questions on a small postcard increases take-up)

  12. Ideas for toolkit continued • Draw and write methods • Move your feet (Children run to different sides of the room to answer the evaluation question) • Physiological measures (Body Mass Index, Waist to hip ratio, Incremental running tests) • Activity counts (Pedometers and Accelerometers)

  13. What I’m doing now and looking forward • Ongoing design and development of the evaluation toolkit • Piloting the evaluation toolkit in one chances4change project • Make refinements to the evaluation toolkit and research methods • Start my main study fieldwork (testing the toolkit out in 6 chances4change projects and/or partnership organisations)

  14. References • Dugdill, L. and Stratton, G. (2007) Evaluating sport and physical activity interventions: A guide for practitioners. University of Salford, UK. • http://www.wellbeingsoutheast.co.uk/chances4change • Ruch-Ross, Keller, D., Miller, N., Basseitz, J. and Melinkovich, P. (2008). Evaluation of community-based health projects: The healthy tomorrows experience. Paediatrics, 122(3), 564-572 • Seppänen-Järvelä, R. (2004). The meaning assigned to evaluation by project staff. Evaluation. 10(4); 430–39 • Stratton, G., Ridgers, N.D., Gobbii, R. and Tocque, K. (2005). Physical activity exercise, sport and health: Regional mapping for the north-west. http://www.nwph.net/pad/ • Taut, S.M., and Alkin, M.C. (2003). Program Staff Perceptions of Barriers to Evaluation Implementation. American Journal of Evaluation. 24; 213 • Van Sluijs, E.M.F., McMinn, A.M., and Griffin, S.J. (2007). Effectiveness of interventions to promote physical activity in children and adolescents: systematic review of controlled Trials. British Medical Journal. http://www.bmj.com/cgi/reprint/335/7622/703

More Related