Local Exercise Action Pilots The National Evaluation General Lessons Learnt So Far!

1. Local Exercise Action PilotsThe National EvaluationGeneral Lessons Learnt So Far! 27-28 May 2004: Andy Pringle leeds metropolitan university

2. Who Am I? • Andy Pringle • Carnegie Faculty of Sport & Education. • Leeds Metropolitan University. • Senior Lecturer in Active Lifestyles and Health. • Project Manager for the National Evaluation of LEAP: Local Exercise Action Pilots.

3. Outline • What is LEAP? • General overview of the National Evaluation Framework for LEAP. • General lessons that have been learnt in developing and managing the National Evaluation Framework. • These are generic and apply to whatever population physical activity interventions are targeted including older people. • Some suggestions of what we would do differently next time around.

4. What is LEAP? • Local Exercise Action Pilots: • This is a £2.6 million contract awarded by Department of Health, Sport England and The Countryside Agency in 2003 to 10 Primary Care Trusts (PCTs/sites) to deliver and evaluate physical activity interventions. • There are 10 pilot sites from the North East to the South West of England using a variety of physical activity interventions which are being evaluated. • LEAP will run for two years.

5. The Interventions • The interventions follow 7 themes: • 1. Exercise Referral • 2. Classes and Groups • 3. Peer Mentoring • 4. Campaigns and Directories • 5. Motivational Interviews • 6. Outdoors and Transport • 7. Training Leaders and Coordinators • with a range of physical activity target groups. • Including older people.

6. What is the Role of leeds metropolitan university?

7. What is the National Evaluation Framework? • Leeds Met has the contract to evaluate LEAP working in partnership with other organizations and consultants. • The basic aim of the National Evaluation Framework is to establish: • Which community physical activity interventions are effective in getting people physically active? • Also looking to find out other information as well.

8. NATIONAL EVALUATION Leeds Metropolitan University + Matrix MESO EVALUATION • The National Evaluation Framework MACRO EVALUATION MICRO EVALUATION LEAP SURVEY MORI + BHF GENERIC QUESTIONS 1. Physical Activity 2. Cost Effectiveness 3. What works, why and how: providers perspective 4. Participants Perspective INTERVENTION THEMES 1. Exercise Referral 2. Classes and Groups 3. Peer Mentoring 4. Campaigns and Directories 5. Motivational Interviews 6. Outdoors and Transport 7. Training Leaders and Coordinators LOCAL EVALUATIONS CASE STUDIES EACH INTERVENTION Run by 10 PCTs Supported by Leeds Met Team Survey Pre (Nov 2003) and Post (Nov 2005) LEAP Interventions 5 LEAP PCTs Dudley Great Yarmouth North Kirklees Wandsworth West Cornwall Comparison Group

9. Three Levels • 1. MACRO 2. MESO 3. MICRO • 1. The MACRO evaluation comprises two elements: • 1. MORI Survey • 2. Case Study data collection that addresses a set of generic questions common to the evaluation of all interventions. • 2. The MESO evaluation comprises seven intervention themes where interventions across the 10 participating PCTs can be compared using data generated by addressing theme specific evaluation questions. • 3. The MICRO evaluation comprises the 10 separate local evaluations incorporating a case study of each intervention.

10. Macro 1. The MORI Survey • Macro Level 1. An independent MORI survey. • Telephone survey of 800 participants conducted in five of the PCTs before (November and December 2003), • and after (November and December 2005), the two years of physical activity intervention.

11. The MORI Survey • 10 minute telephone questionnaire. • The survey instrument covers questions on: • Physical activity category. • Awareness of local physical activity campaigns. • Knowledge and attitudes to physical activity. • Social climate, intention to be active. • Demographics.

12. Macro 2. A Case Study in the 10 PCTS/Sites • The majority of the information for the N.E.F is being collected at the PCT level by PCTS and associates. • A common set of questions across all interventions where appropriate. • 1. Baseline and change in physical activity status. • 2. Effect of the intervention on participant’s FITT of participation. • 3. Who the participants are and where are they are from? Contact details & demographics. • 4. Is the activitynew or a replacement physical activity.

13. The National Evaluation Framework • 5. Cost analysis of planning, designing and delivering the LEAP interventions • Data collection on inputs (e.g.. personnel), outputs (e.g.. classes) and outcomes (e.g.. change in physical activity behavior). • Working in collaboration with MATRIX CR. • 6. Qualitative evaluation of planning, delivery and evaluation of the interventions: • What works, why and how? • Undertaken by the Leeds Met Field Researchers.

14. What are the lessons that have been learnt….so far? What action could be taken to prevent and manage these issues in the future?

15. Collaboration • It is clear that there are some “first rate” examples of collaborative working between the PCTs/Pilot Sites and Leeds Met in the development of the Evaluation Framework. • However there are examples of less enthusiasm for the evaluation (in its current context). • The perception of what evaluation constitutes and how much work is needed: • Resource base available. • Skills available for evaluation.

16. Specific information • There are examples of where the PCTS have under-estimated/been unclear of the size of the work commitment of the evaluation component for LEAP. • Action: Provision of clear and specific information of what the PCTs commitment is at the outset is needed. • Specifically what the magnitude of the task is?

17. Clarity on Roles and Responsibilities • The specific types of interventions to be evaluated. • Finalizing the design of the evaluation framework (to Inc tools, dates of data capture. • Leeds Met to be involved in the site selection.

18. Resources: Money & Time • Resources will always be a contentious issue. • Health Promotion writers talk of the need to ensure that evaluation is supported with the necessary resources (Green & Tones, 2004, Naidoo & Wills, 2000). • It is important that evaluation frameworks are well supported with resources so that the the best possible job can be done.

19. Other Issues • Choice of tools: • Particularly for older measuring physical activity levels in people. • IPAQ: Recall of information. • Where English is not spoken. • Objective measures of physical activity: • Anxiety of some of the sites. • Local Ethical Clearance. • Tensions between the job of evaluation and delivery of physical activity interventions: Clear Information.

20. For further information: • Andy Pringle • Senior Lecturer: Active Lifestyles & Health • Carnegie Faculty of Sport & Education • Leeds Metropolitan University • Fairfax 212, Beckett Park Campus • LEEDS LS6 3QS. • A.Pringle@LEEDSMET.AC.UK • 0113 283 2600 Ext.7409. • WWW.LEEDSMET.AC.UK

21. leeds metropolitan university Andy Pringle Senior Lecturer: Active Lifestyles & Health