1 / 28

Bring me the Evidence!

National Obesity Observatory The Standard Evaluation Framework Dr Louisa Ells louisa.ells@nepho.org.uk Dr Monica Dent mdent@nhs.net. Bring me the Evidence!. One question…. If evaluation was an animal… what would it be?. Has it achieved its objectives?. Has it worked?. Does it work?.

bree
Download Presentation

Bring me the Evidence!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National Obesity ObservatoryThe Standard Evaluation FrameworkDr Louisa Ellslouisa.ells@nepho.org.ukDr Monica Dent mdent@nhs.net

  2. Bring me the Evidence!

  3. One question….. If evaluation was an animal… what would it be?

  4. Has it achieved its objectives? Has it worked? Does it work? What happened? Was it effective? Project Do I need a double blind randomised control trial? Could it work? What questionnaire should I use?

  5. Standard Evaluation Framework (SEF) for weight management interventionswww.noo.org.uk/SEF

  6. Aim of the SEF: To support high quality, consistent evaluation of weight management interventions in order to increase the evidence base

  7. Objectives of the SEF: • Increase the proportion of weight management interventions that are evaluated • Increase the quality of evaluations • Develop a core dataset to increase the consistency and comparability of evaluations • Provide wider evaluation support

  8. Target audience for the SEF: • PCT commissioners (to support service specifications) • PCT & DH obesity leads • Organisations running interventions • External evaluators In some regions (NE, NW), the SEF is a required element of all new weight management programmes

  9. Why use the SEF? Evidence-based interventions Number of interventions Speculative interventions Time

  10. What interventions should it apply to? • Any intervention that explicitly sets out to manage or reduce body weightas an intermediate or long-term health outcome (including the primary prevention of weight gain) • Projects focusing on diet, physical activity, or both in combination • Intended to be applicable to a range of approaches including interventions conducted with individuals on a one-to-one basis, or with people in groups; and in clinical or community settings.

  11. The SEF is not intended for: • Medical treatment e.g. bariatric surgery & prescribing • Wider environmental interventions e.g. changes to the built environment (although the SEF could support the evaluation of such interventions alongside wider evaluation criteria)

  12. Consultation Processes: Developed in consultation with academic experts, representatives from Public Health Observatories, Primary Care Trusts, Government Offices, Local Authorities and other relevant organisations and bodies Initial list of criteria consulted on using an online questionnaire - followed up by individual interviews Two consultation meetings to finalise criteria and guidance. NB the SEF itself will also be evaluated.

  13. What is in the SEF document? • An introduction to the concepts of evaluation • A list of data collection criteria divided into ‘essential’ and ‘desirable’ • A guide to using the criteria and collecting the data

  14. Introduction to Evaluation: • What is evaluation and why do it? • What are the different types of evaluation? • What are the differences between evaluation, audit and monitoring? • What sorts of information need to be collected to carry out evaluations? • What sorts of issues need to be considered as part of an evaluation?

  15. SEF Criteria: • Programme details e.g. name of intervention, aims/objectives • Demographics of individual participants e.g. age, sex, ethnicity • Baseline data e.g. height/weight, dietary and physical activity behaviours • Follow-up data (impact and process evaluation) • Analysis and interpretation

  16. Supporting SEF guidance: • Details about why criteria have been categorised as ‘essential’ or ‘desirable’ • What the data can and cannot tell you • Standardised methods for collecting some of the data • Suggested standardised definitions for data • Suggested data collection tools • Signposting to further guidance and useful information

  17. Recommended Physical Activity Measurement Tools: • 4 instruments met the following criteria • Relatively easy and practical to administer • Mean difference between energy estimated by self-report and doubly labelled water <40% • Instrument should have been used in a trial of exercise promotion and detected a significant difference between intervention and control participants • Population in the validity study generalisable to wider population Appendix to SEF by Dr Melvyn Hillsdon

  18. Recommended Physical Activity Measurement Tools: • International Physical Activity Questionnaire (IPAQ) long version • 7day recall • Valid in 15-69 year olds only • Stanford 7-day recall • Self report for sleep, moderate, hard and very hard activities • Valid in adults only

  19. Recommended Physical Activity Measurement Tools: • 7-day physical activity diary • Self-completion diary ‘ticking’ 15min blocks of activity • Valid in adults only • New Zealand Physical Activity Questionnaire • 7day Self-completion recall, using prompt cards • Valid in middle aged adults only

  20. Limitations: • No existing physical activity measure perfectly meets all criteria • Lack of suitable measures for children • No gold standard for measuring sedentary behaviours • Most measures have quite complex scoring systems

  21. Evaluating the SEF: • How we are evaluating the SEF: • SEF user survey • Qualitative interviews/focus groups with regional leads/programme leads • How you can help? • Take part in the user survey/interviews • Feedback your comments & suggestions • Measurement tools used & why • What measurement tools you’ve used for children • Any problems using the SEF for physical activity interventions • Anything you would like to see in the revised SEF

  22. Future SEF products: • Stand alone SEF criteria • Example case studies • Template for data collection • Online audit (currently taking place in the NE), national audit under development.

  23. Questions? info@noo.org.uk

  24. Interactive Workshop • Aims: • Share how you plan to use the SEF in your locality. • Discuss using the SEF for an intervention(s) running or due to run in your locality and identify: • Possible barriers/difficulties to using the SEF. • How the SEF can be used to improve equality and diversity. • Possible solutions to these barriers and how NOO can best support these. • The advantages the SEF would bring to this intervention.

  25. Workshop Feedback: • Each group to report back: • How is the SEF going to be used in your locality? • How could the SEF improve equality and diversity. • What possible barriers are there to using the SEF? • How could these barriers be overcome? • What could NOO provide to support use of the SEF further?

More Related