1 / 42

The RRFSS Evaluation Framework 2005 Background Information and Progress to Date

The RRFSS Evaluation Framework 2005 Background Information and Progress to Date. Presenters Amira Ali Jane Hohenadel Kate O’Connor Deborah Radcliffe June 22, 2005. Group Members.

glenna
Download Presentation

The RRFSS Evaluation Framework 2005 Background Information and Progress to Date

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The RRFSS Evaluation Framework 2005Background Information and Progress to Date Presenters • Amira Ali • Jane Hohenadel • Kate O’Connor • Deborah Radcliffe June 22, 2005

  2. Group Members • RRFSS Evaluation Group: Amira Ali, Elaine Hector, Jane Hohenadel, Irene Koren, Kathy Moran, Karen Moynagh, Lynne Russell, Louisa Wong • PHRED RRFSS Evaluation Group: Jane Hohenadel, Adriana Newbury, Kate O’Connor, Louise Picard, Deborah Radcliffe

  3. Outline of Presentation • Link to strategic planning session • Link to previous evaluation (2002) • Evaluation Framework – RRFSS Evaluation 2005 • Tasks and Timelines

  4. History • Project was identified in March 2004 • Proposal developed in May 2004 and an Evaluation committee was formed with representation from each of the RRFSS regional working groups • The PHREDs were identified as partners in the project – proposal submitted to them in October 2004 • Terms of reference for the RRFSS group and the PHRED group were finalized in December 2004

  5. Link to Strategic Planning Session

  6. Strategic Planning Session • Strategic Planning Session in December 2004 • To develop strategic statements (vision, mission, values, goals, strategies) that will guide the collective RRFSS work over the next 3 years • To determine the specific objectives and activities for the next year • To create an action document summarizing the directions set

  7. Strategic Planning Session • Vision: RRFSS envisions that all decisions within the public health system to promote and protect health and wellbeing and prevent adverse health events are informed by valid, timely and relevant health intelligence • Mission: Ontario’s RRFSS is a flexible, timely and responsive surveillance system designed to meet local Public Health intelligence needs and address information, geographic and time-related data gaps

  8. Strategic Planning Session • Four strategies were identified: • Advocacy • Communication • Governance • Quality Improvement and Development

  9. Strategic Planning Session • Strategy: Quality Improvement and Development Priority Activity: “Conduct the RRFSS 2005 Evaluation to identify areas of the surveillance system in need of improvement” The full report “Strategic Plan for Action” can be viewed at: www.rrfss.on.ca

  10. Link to RRFSS Evaluation 2002

  11. RRFSS Evaluation 2002 • Covered first year of operation (2001) • Number of health units ranged from 6 to 12 during the evaluation period • Purpose: • Inform change to system procedures • Contribute to continuous quality improvement • Act as a baseline and template for future evaluations The full evaluation report can be viewed at: www.rrfss.on.ca

  12. RRFSS Evaluation 2002 • Evaluation Framework covered 4 areas: • Process Effectiveness • Collaboration Effectiveness • Usefulness • Cost Effectiveness

  13. RRFSS Evaluation 2002 • Performance criteria linked to following outcomes expected from RRFSS: • Increased local awareness of surveillance information • Increased involvement in generating appropriate, accurate information • Increased practice of evidence-based program planning • Demonstrated or potential cost benefit due to RRFSS

  14. RRFSS Evaluation 2002 Fourteen Key Issues Identified(grouped as below for 2005 evaluation) • Local Resources (2) • Centralization Issues • Provincial Implications (3) • Implications for ISR (4) • Process Considerations (5)

  15. Key Issues from 2002 EvaluationLocal Resources • Adequate staff/resources for data analysis and making data useable • RRFSS coordinators overwhelmed by work at the local level

  16. Key Issues from 2002 EvaluationCentralization – Provincial Implications • Need for stability and support for the central work of the RRFSS partnership • Sustaining funding for existing and new participants (smaller health units in particular); includes funding for survey administration and analysis • Provincial representation of data

  17. Key Issues from 2002 Evaluation Centralization – Implications for ISR • Increasing the sample size for a health unit • Access to ISR, decision-making, workload and resources • Improved timeliness of data from ISR • ISR Ability to handle increase in HU participation

  18. Key Issues from 2002 EvaluationProcess Consideration Issues • Increasing use of data in program planning and evaluation in the health units • Limited new question/module capacity under current questionnaire • Process for revising old modules and developing new modules • Upgrading RRFSS website • Sharing data among organizations

  19. RRFSS Evaluation 2005

  20. 2005 Evaluation Framework Objectives • To document progress made in the key areas identified in the 2002 Evaluation • To document some of the “best practices” used by Health Units • To envisage what it would take to get RRFSS on board for all Ontario Health Units and how it could work as a provincial system

  21. 2005 Evaluation Framework Refinement of the Key Issues • To address the evaluation objectives, we began with what were identified as Key Issues in the 2002 Evaluation • Evaluation group members were asked to review each of those 14 Key issues, and: • report on progress • whether it was still an issue now • if so, how important it was for the 2005 evaluation

  22. 2005 Evaluation Framework Refinement of the Key Issues • It was felt that seven issues had been largely addressed, and that the 2005 evaluation needed only to document their evolution since 2002 • The issues that we felt were addressed: • increasing the sample size for a health unit • improved timeliness of data from ISR • access to ISR, decision-making, workload and resources, and • limited new question/module capacity

  23. 2005 Evaluation Framework Refinement of the Key Issues • The issues that we felt were addressed (cont’d): • limited new questionnaire/module capacity under current questionnaire • process for revising old modules and developing new modules • upgrading RRFSS website

  24. 2005 Evaluation Framework Refinement of the Key Issues • The remainder were felt to still be issues for the RRFSS partnership and should remain in the 2005 evaluation • The outstanding issues were refined, defined and expressed as evaluation questions

  25. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Local Resources Question Do HUs have adequate staff and resources for regional and local administrative tasks of RRFSS? Data Collection Tool Questionnaire Source of Data RRFSS reps, MOH of participating HU

  26. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Local Resources Question Do HUs have adequate staff and resources locally for data analysis and making data useable? Data Collection Tool Questionnaire Source of Data RRFSS reps, MOH of participating HU

  27. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Local Resources Question Are local RRFSS reps able to meet the local work demand for RRFSS module development? Data Collection Tool Questionnaire Source of Data RRFSS reps, MOH of participating HU, program and research staff, RRFSS coordinator, RRFSS Evaluation Group, ISR

  28. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Local Resources Question Are local RRFSS reps able to meet the local work demand for RRFSS module development? Data Collection Tool Questionnaire Source of Data RRFSS reps, MOH of participating HU, program and research staff, RRFSS coordinator, RRFSS Analysis Group, ISR

  29. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Local Resources Question Are local RRFSS reps able to meet the local work demand for RRFSS module review? Data Collection Tool Questionnaire Source of Data RRFSS reps, MOH of participating HU, program and research staff, RRFSS coordinator, RRFSS Analysis Group, ISR

  30. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Centralization Issues: Provincial Implications Question What resources are needed currently to support and to ensure the stability of the provincial and administrative work of the RRFSS Partnership? Data Collection Tool Questionnaire Source of Data RRFSS reps, MOH of participating HU, ISR, RRFSS Coordinator

  31. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Centralization Issues: Provincial Implications Question What funds are required to enable continued participation of HUs currently participating in RRFSS and allow non-participating HUs to join? Data Collection Tool Key Informant Interviews Source of Data RRFSS reps, MOH of all HUs, ISR, MOHLTC

  32. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Centralization Issues: Provincial Implications Question Is provincial representation of data a desired product of RRFSS? Data Collection Tool Key Informant Interviews Source of Data RRFSS reps, MOH of all HUs, MOHLTC, PHRED, Chief MOH, external partners

  33. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Centralization Issues: Provincial Implications Question What would it take to extend RRFSS to all Ontario HUs? Data Collection Tool Key Informant Interviews Source of Data RRFSS reps, epidemiologists in non-participating HUs, MOHLTC, ISR, Chief MOH, MOH of all HUs

  34. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Process Considerations Question To what extent are RRFSS data being used in PP&E in the health units? Data Collection Tool Questionnaire, Case Examples Source of Data RRFSS reps, program and research staff

  35. 2005 Evaluation Framework Evaluation Questions for Outstanding Issues Issue Process Considerations Question To what extent are provincial RRFSS data being shared among agencies and organizations? Data Collection Tool Questionnaire, Key Informant Interview Source of Data RRFSS Coordinator, ISR, RRFSS reps

  36. Evaluation Framework for RRFSSEvaluation Questions for Outstanding Issues Issue Process Considerations Question To what extent are health unit specific RRFSS data being shared among agencies and organizations? Data Collection Tool Questionnaire Source of Data RRFSS reps

  37. 2005 Evaluation FrameworkEvaluation Question for Objective 2 Issue Process Considerations Question What Success Stories can be identified for RRFSS? Data Collection Tool Workshop, Questionnaire Source of Data RRFSS reps, program and research staff

  38. 2005 Evaluation Framework Evaluation Question Issue Process Considerations (Parked Issue) Question What is the quality of RRFSS data? Data Collection Tool Questionnaire Source of Data RRFSS reps, ISR, partners

  39. Tasks and Timelines

  40. RRFSS Evaluation Tasks • Finalize development of research tools. • Questionnaire • Focus groups • Key informant interviews • Document reviews • Data collection: • RRFSS Workshop-- “Success stories” • Primarily August and September • Data Analysis • Draft and finalize the report

  41. RRFSS Evaluation Timelines

More Related