1 / 28

Bring me the Evidence!

National Obesity Observatory The Standard Evaluation Framework Kath Roberts kath.roberts@noo.org.uk/ katharine.roberts@empho.nhs.uk Nick Cavill nick.cavill@cavill.net. Bring me the Evidence!. One question…. If evaluation was an animal… what would it be?. Finally.

camilla-cardenas
Download Presentation

Bring me the Evidence!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National Obesity ObservatoryThe Standard Evaluation FrameworkKath Roberts kath.roberts@noo.org.uk/katharine.roberts@empho.nhs.ukNick Cavillnick.cavill@cavill.net

  2. Bring me the Evidence!

  3. One question….. If evaluation was an animal… what would it be?

  4. Finally........

  5. Has it achieved its objectives? Has it worked? Does it work? What happened? Was it effective? Project Do I need a double blind randomised control trial? Could it work? What questionnaire should I use?

  6. Standard Evaluation Framework (SEF) for weight management interventionswww.noo.org.uk/SEF

  7. Aim of the SEF: To support high quality, consistent evaluation of weight management interventions in order to increase the evidence base

  8. Objectives of the SEF: • Increase the proportion of weight management interventions that are evaluated • Increase the quality of evaluations • Develop a core dataset to increase the consistency and comparability of evaluations • Provide wider evaluation support

  9. Target audience for the SEF: • PCT commissioners (to support service specifications) • PCT & DH obesity leads • Organisations running interventions • External evaluators In some regions (NE), the SEF is a required element of all new weight management programmes?

  10. Why use the SEF? Evidence-based interventions Number of interventions Speculative interventions Time

  11. What interventions should it apply to? • Any intervention that explicitly sets out to manage or reduce body weightas an intermediate or long-term health outcome (including the primary prevention of weight gain) • Projects focusing on diet, physical activity, or both in combination • Intended to be applicable to a range of approaches including interventions conducted with individuals on a one-to-one basis, or with people in groups; and in clinical or community settings.

  12. The SEF is not intended for: • Medical treatment e.g. bariatric surgery & prescribing • Wider environmental interventions e.g. changes to the built environment (although the SEF could support the evaluation of such interventions alongside wider evaluation criteria)

  13. Consultation Processes: Developed in consultation with academic experts, representatives from Public Health Observatories, Primary Care Trusts, Government Offices, Local Authorities and other relevant organisations and bodies Initial list of criteria consulted on using an online questionnaire - followed up by individual interviews Two consultation meetings to finalise criteria and guidance. NB the SEF itself will also be evaluated.

  14. What is in the SEF document? • An introduction to the concepts of evaluation • A list of data collection criteria divided into ‘essential’ and ‘desirable’ • A guide to using the criteria and collecting the data

  15. Introduction to Evaluation: • What is evaluation and why do it? • What are the different types of evaluation? • What are the differences between evaluation, audit and monitoring? • What sorts of information need to be collected to carry out evaluations? • What sorts of issues need to be considered as part of an evaluation?

  16. SEF Criteria: • Programme details e.g. name of intervention, aims/objectives • Demographics of individual participants e.g. age, sex, ethnicity • Baseline data e.g. height/weight, dietary and physical activity behaviours • Follow-up data (impact and process evaluation) • Analysis and interpretation

  17. Supporting SEF guidance: • Details about why criteria have been categorised as ‘essential’ or ‘desirable’ • What the data can and cannot tell you • Standardised methods for collecting some of the data • Suggested standardised definitions for data • Suggested data collection tools • Signposting to further guidance and useful information

  18. Recommended Physical Activity Measurement Tools: • 4 instruments met the following criteria • Relatively easy and practical to administer • Mean difference between energy estimated by self-report and doubly labelled water <40% • Instrument should have been used in a trial of exercise promotion and detected a significant difference between intervention and control participants • Population in the validity study generalisable to wider population Appendix to SEF by Dr Melvyn Hillsdon

  19. Recommended Physical Activity Measurement Tools: • International Physical Activity Questionnaire (IPAQ) long version • 7day recall • Valid in 15-69 year olds only • Stanford 7-day recall • Self report for sleep, moderate, hard and very hard activities • Valid in adults only

  20. Recommended Physical Activity Measurement Tools: • 7-day physical activity diary • Self-completion diary ‘ticking’ 15min blocks of activity • Valid in adults only • New Zealand Physical Activity Questionnaire • 7day Self-completion recall, using prompt cards • Valid in middle aged adults only

  21. Limitations: • No existing physical activity measure perfectly meets all criteria • Lack of suitable measures for children • No gold standard for measuring sedentary behaviours • Most measures have quite complex scoring systems

  22. Evaluating the SEF: • How we are evaluating the SEF: • SEF audit tool (currently piloted) • Qualitative interviews with regional leads • Qualitative interview with programme leads • How you can help? • Use the audit tool • Take part in interviews • Feedback your comments & suggestions • Measurement tools used & why • What measurement tools you’ve used for children • Any problems using the SEF for physical activity interventions • Anything you would like to see in the revised SEF

  23. Future SEF products: • Stand alone evaluation guide • Stand alone criteria tables • Example case studies • Excel template for audit data • Excel template for individual study data • Online audit (currently taking place in the NE), national audit under development.

  24. Exercise: • Each group to report back: • How is the SEF going to be used in your locality? • What possible barriers are there to using the SEF? • How could these barriers be overcome? • What could NOO provide to support use of the SEF further?

  25. Case Study: Derbyshire FLIP • Locally developed programmes to come together under ‘umbrella’ of Derbyshire FLIP • BMI • Matching objectives with outcomes • Clear referral routes • Validated dietary and PA questionnaires • Bringing 3 programmes together with common ‘weight management’ objectives • Data collection and analyses

  26. Questions? info@noo.org.uk

More Related