1 / 35

Evidence-based IP in Hong Kong: State-of- the-art evidence from local programmes Kevin Chan

Evidence-based IP in Hong Kong: State-of- the-art evidence from local programmes Kevin Chan Project Associate Network for Health & Welfare Studies Department of Applied Social Sciences Hong Kong Polytechnic University Scientific Officer

galena
Download Presentation

Evidence-based IP in Hong Kong: State-of- the-art evidence from local programmes Kevin Chan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evidence-based IP in Hong Kong: State-of- the-art evidence from local programmes Kevin Chan Project Associate Network for Health & Welfare Studies Department of Applied Social Sciences Hong Kong Polytechnic University Scientific Officer Hong Kong Childhood Injury Prevention & Research Association November 1, 2003

  2. What are the outcomes of evaluation? • Health outcome • Reduction of injuries in targeted population • Requires longitudinal observation • Using intermediate outcomes to approximate potential health outcome

  3. What are the outcomes of evaluation? • Intermediate outcomes • Knowledge related to hazard identification • Knowledge related to safety behavior • Observed / reported safety behavior • Observed / reported environmental modification

  4. Research designs for evaluation • Different designs to accommodate programmes in school and home settings: • Cluster randomized trial for school-based programme (IPPS) • Randomized Control trial for home-based programme (VHVP)

  5. Synopsis of IPPS findings Design: Randomized controlled trial Targets: P.5 & P.6 students, primarily from the Greater Shatin District (including Tai Wai, Shatin, Ma On Shan) N=1311 students Intervention group = 686 (5 schools); Control group = 625(3 schools) Outcome = Improvement in self-reported injury prevention knowledge between intervention and control schools on repeated measures

  6. Synopsis of VHVP findings • Procedure and material: • Intervention group – Annual hospital visit to A&E department, IP curriculum and teaching aids, in-school demonstration by IPPS staffs • Control group - Nil

  7. Synopsis of IPPS findings • Major domains of IP knowledge tested in IPPS • Hazard identification • Crisis handling • First-aid • Home safety • Road safety • Poisoning • Electrocution

  8. Synopsis of IPPS findings • Observed between-group difference of change in IPPS knowledge test score (Pre-intervention to Post-intervention) is 3.06(95% C. I. = 2.39, 3.74), and the difference is statistically significant (T-test = 8.9, df=1309, p<0.01)

  9. Synopsis of IPPS findings • Intervention group students are 2.7 times more likely to show improvement on IP knowledge over time (95% C. I. = 2.1, 3.5) • Girls in intervention schools show higher improvement in IP knowledge (OR=3.4, 95% C. I. =2.3, 4.9) than their boys counterparts (OR=2.2, 95% C. I. = 1.5, 3.2)

  10. 參與學校(X2)及對照學校(Y2)比較測試成績 (第二階段) 總分： 94

  11. 參與學校(X2)及對照學校(Y2)比較整體成績之進步比率(第二階段)參與學校(X2)及對照學校(Y2)比較整體成績之進步比率(第二階段)

  12. 參與學校(X2)及對照學校(Y2)比較研究選項之進步比率(第二階段)參與學校(X2)及對照學校(Y2)比較研究選項之進步比率(第二階段)

  13. Synopsis of IPPS findings • Process effectiveness of IPPS: • 70% of participating teachers expressed needs for IP in school and curriculum • 90%of participating teachers believed that IPPS fits into their existing curriculum and students profiles

  14. Synopsis of VHVP findings Design: Randomized controlled trial Targets: New-born to 3-year-old children admitted into A&E department of Princess Margaret Hospital or Prince of Wales Hospital for an Unintentional Residential Childhood Injury (URCI) episode N=76 (Interim results from 194 recruited families) Intervention families = 40 Control families = 36

  15. Synopsis of VHVP findings • Procedure and material: • Control group - Printed education package developed locally by the research team • Intervention group - Educational material + 4 quarterly home visits and monthly telephone follow-ups

  16. Synopsis of VHVP findings • Evaluation Measures: • A 20-item 5-point scale on injury beliefs (Injury Belief Questionnaire - IBQ) & a 51-item checklist on behavioral and environmental hazards in the household (Household environment checklist - HEC) • Families from both arms were assessed on all instruments except that control families were given no further advice or follow-up

  17. Synopsis of VHVP findings • Analyses: • Group difference between intervention and control schools on repeated measures i.e. change in Home Environment Checklist (HEC) safety ratings. • Statistical test: Moses extreme reactions test on change in HEC between groups

  18. Synopsis of VHVP findings Home environment modification Having a first-aid box at home (Control group span = 56, p<0.01) Household rearrangement to avoid staggering furniture layout (Control group span = 36, p<0.01) Testing temperature in microwave oven prepared food (Control group span = 20, p=0.05 Childproofing electrical heating devices in kitchen (e.g. boiler, rice cooker) (Control group span = 20, p=0.05).

  19. Synopsis of VHVP findings Change in injury prevention beliefs: Significant improvement in child supervision (Mann-Whitney U = 460, p=0.03) Task efficacies filleting fishes in children’s meals (Control group span = 62, p=0.03)

  20. Synopsis of VHVP findings Cost-effectiveness of VHVP: Though we have yet to conduct any thorough evaluation at this stage, it is apparent that such initiative is cost-effectiveness even on rough estimation.

  21. Synopsis of VHVP findings Cost-effectiveness of VHVP: At an average annual cost of HKD $274 per beneficiary, the VHVP would cost $323,868 for 1182 children per year, which was the number of 0-3 home injuries in Shatin in 2002. At an average per head cost of $1,536, the estimated healthcare cost incurred is $1,815,552; the avoidable healthcare cost at a preventable fraction of 0.2 would be $363,110.4.

  22. Synopsis of VHVP findings Cost-effectiveness of VHVP: From such estimation, the VHVP already contributed to a annual cost-benefit of $39,242 in the Shatin district alone. It should be noted that the demonstrated cost benefit would elevate much further if the indirect cost (e.g. work day lost by parents) and intangible cost of child injuries (e.g. quality adjusted life year) is accounted for.

  23. Synopsis of VHVP findings Process-effectiveness of VHVP: During the VHVP implementation, we also evaluate the process effectiveness of our volunteer visits. Two major findings were observed.

  24. Synopsis of VHVP findings Process-effectiveness of VHVP: Volunteer commitment, rated by their respective supervisors, were found to be reliable indicator of their service retention (Kendall-tau = 0.332, p=0.03). IP knowledge attainment has a curvilinear effect on household hazards reduction (Kruskal-Wallis test statistics = 8, df = 2, p=0.02).

  25. Beyond findings: Issues in evaluating IP programmes • Major issues in IP programme evaluation beyond present findings: • Type III error in IP programme • Impact on community • Social ecological impact brought by IP programme

  26. Beyond findings: Issues in evaluating IP programmes • Type III error in IP programme: • mistaken assumption that a program was implemented as planned, as well as identify specific components that were implemented poorly or not at all • often undermined in health promotion evaluation • process evaluation is a common tool for minimizing Type III error in health promotion

  27. Beyond findings: Issues in evaluating IP programmes • Type III error in IP programme: • Major relevant issues in process evaluation of VHVP: • Maintenance—keeping participants involved in the programmatic and data collection • Context—aspects of the environment of an intervention • Implementation—the extent to which the program is implemented as designed • Barriers—problems encountered in reaching participants • Contamination—the extent to which participants receive interventions from outside the program and the extent to which the control group receives the treatment • (Linnan & Steckler, 2002)

  28. Beyond findings: Issues in evaluating IP programmes • Impact on community: • Dual role of community-based injury prevention programmes: • Instrumental and constitutive functions • (Labonte, 2001)

  29. Beyond findings: Issues in evaluating IP programmes Impact on community: Instrumentalfunction: Facilitate sustainable injury prevention effort by mobilizing community members and network to achieve the desired health outcome(s) and Constitutive function: Empowering community to identify their own health issues and enhance its capacity to deal with these issues (Labonte, 2001)

  30. Beyond findings: Issues in evaluating IP programmes Social ecological impact brought by IP programme: Ecological change in systems or sub-systems plays an important role in the formation and maintenance of health promotion behavior (McLeroy, 1988; Davies, 2002)

  31. Beyond findings: Issues in evaluating IP programmes Social ecological impact brought by IP programme: Changes that IP programme brought into the social environment (i.e. school / family) are critical to the understanding of IP behavior in a practice setting.

  32. Beyond findings: Issues in evaluating IP programmes • Social ecological impact brought by IP programme: • Examples: • How school administration / structure / curriculum adapt to the IPPS? What is the “uptake rate” of IPPS by different students / teachers / schools? • How family members, along with the home visiting volunteer, work together to achieve goals recommended in the VHVP?

  33. Subtitle • Conclusion • Findings up to date have lent support to the effectiveness of these featured programmes. • Preliminary analysis suggested that these initiatives could be cost-effective. • Revealed in process evaluation, strengthening volunteer commitment & optimizing IP knowledge during visit have direct impact on injury prevention practices.

  34. Subtitle • Conclusion (Cont’d) • Further analyses on process evaluation is to be developed for the minimization of Type III error • Community-based IP benefits not only on a programme level, but also on a community level in terms of capacity building and empowerment • Follow-up studies on systemic changes brought by the feature programmes are recommended

  35. Acknowledgement: The Hong Kong Childhood Injury Prevention & Research Association Quality Education Fund

More Related