Current State of the Art in Economic Evaluations of OSH Interventions

1. Current State of the Art in Economic Evaluations of OSH Interventions Emile Tompa ECOSH and ROWER Workshop September 17-18, 2009

2. Outline 1. Introduction 2. Overview of challenges in the economic evaluation of OSH interventions 3. Substantive findings from a systematic review 4. Methods issues identified through the review 5. Distinguishing between a business case and an economic evaluation 6. Suggestions for the framing of OSH economic evaluations 7. Developing a reference case 8. Discussions

3. Introduction: Economic Evaluation Initiative at IWH • Systematic review of workplace-based OSH studies with economic evaluations • Methods text for researchers • Software for workplace parties • Training workshops for workplaces and OSH specialists

4. Introduction: Challenges in OSH Economic Evaluation • Complex OSH policy arena and labour legislation • Multiple stakeholders and conflicting incentives/priorities • Differences in the perceptions of health risks • Differences in what ought to be counted as costs and benefits • Distribution of costs and consequences across multiple stakeholders • Multiple providers of indemnity insurance • Industry specific human resources practices • Absence of good guidelines in how to deal with the above

5. Introduction: Methods Development Work • Some methods development work being done • WHO/NIOSH workshop • Koopmanschap, Brouwer and colleagues: friction costs of absenteeism and presenteeism • Pauly and colleagues: direct and indirect cost of absenteeism • Uegaki and colleagues: health related productivity cost • Guzman and colleagues: worker costs of work injury • Tompa and colleagues: methods text

6. Systematic review substantive findings: overview • Broad search of journal database, other systematic reviews, request from content experts • Broad inclusion criteria– all workplace-based intervention studies with economic evaluations, some exclusions • 12,903 titles and abstracts citations • Retained 72 peer-reviewed intervention evaluations • 33 retained for evidence synthesis

7. Systematic review substantive findings: citations and abstracts search results Medline 6,381 EMBASE 6,696 BIOSIS 2,568 BSP 687 Ergo Abs 25 Other 199 Merged Database 12,903 article 67 studies met inclusion criteria contained 72 intervention evaluations

8. Systematic review substantive findings: ergonomics and other MSD injury prevention Administrative and support • 2 high quality, 1 medium quality, 5 low quality • moderate evidence Health care • 4 medium quality, 7 low quality • moderate evidence Manufacturing and warehousing • 3 high quality, 2 medium quality, 4 low quality • strong evidence Transportation • 1 high quality, 2 medium quality • moderate evidence

9. Systematic review substantive findings:other intervention types Health care • Occupational disease prevention • 3 medium quality, 2 low quality • moderate or limited evidence Manufacturing and warehousing • multi-faceted interventions • 2 medium quality, 2 low quality • limited evidence of negative findings or mixed evidence Multi-sector • Disability management • 4 high quality, 1 low quality • strong evidence

10. Systematic review methods issues: summary Two main findings • Few workplace-based intervention studies undertake economic analyses • Of those that do, mixed bag of methodological approaches and quality Methodological shortcomings • Study design and related factors • Measurement and analytic factors • Computational and reporting factors

11. Systematic review methods issues: some details • Weak study designs, primarily before/after without controls • Disconnect between effectiveness and economic analysis • Reliance on workers’ compensation expenses as the sole outcome measure • Perspective often not explicitly stated • Failure to adjust monetary values for inflation and time preference • Reliance on questionable assumptions with no sensitivity analysis • Scant reporting of the details such as context, firm size, sample size, time period, details of the intervention

12. Business case versus economic evaluation (1) Business case for OSH interventions • Analysis designed to support organizational decision making • Generally takes a firm’s point of view • Not only about improving financial performance • Can also be about “getting health and safety right,” improving productivity, avoiding certain costs, meeting client demands, and improving staff morale • Comprehensive quantitative analysis costs and consequences not always necessary

13. Business case versus economic evaluation (2) Economic evaluation of OSH interventions • May include the business case • Has more comprehensive consideration of costs and consequences • Often considers resource implications to stakeholders other than just the firm • Tests robustness of results to key assumptions • Comprehensive reporting of intervention contexts, execution, analysis and results

14. Suggestions for framing an economic evaluation (1) Three framework principles proposed as a starting point for developing a reference case (Culyer et al., 2008) 1) The primary objective of OSH interventions is to enhance the expected health-related welfare of individuals in the workplace 2) The perspective of particular evaluative studies will be determined in conjunction with the relevant stakeholders, supplemented where necessary by analyses that incorporate significant external effects

15. Suggestions for framing an economic evaluation (2) 3) Economic evaluations should, in addition to considering efficiency, identify potential equity issues of significance in conjunction with stakeholders, and always present results in a way that reveals how the incidence of costs and benefits falls both immediately and after any predictable market adjustments have been made.

16. Developing a reference case • Need for set of conventions for framing, analysis and presentation of OSH economic evaluation • Health care reference case proposed by Gold et al. (1996) and National Institute for Clinical and Evaluative Sciences (2004) • To make it work, need ongoing dialogue between OSH researchers, workplace parties, policymakers, and other stakeholders • ECOSH and ROWER workshop is an excellent place to begin such dialogue

17. References • Tompa E, Dolinschi R, de Oliveira C. 2006. Practice and potential of economic evaluation of workplace-based interventions for occupational health and safety. Journal of Occupational Rehabilitation, 16, 375-400. • Tompa E, Dolinschi R, de Oliveira C, Irvin E. 2007. Final Report on a Systematic Review of OHS Interventions with Economic Evaluations. Toronto: Institute for Work & Health. • Tompa E, Culyer A, Dolinschi R, eds. 2008. Developing Good Practice in the Economic Evaluation of Workplace Interventions for Health and Safety. Oxford: Oxford University Press. • Tompa E, de Oliveira C, Dolinschi R, Irvin E. 2008. A systematic review of disability management interventions with economic evaluations. Journal of Occupational Rehabilitation, 18, 15-26. • Tompa E, Dolinschi R, de Oliveira C, Amick B, Irvin E. (in press). A Systematic Review of Workplace Ergonomic Interventions with Economic Analyses. Journal of Occupational Rehabilitation. • Tompa E, Dolinschi R, de Oliveira C, Irvin E. (in press). A systematic review of occupational health and safety interventions with economic analyses. Journal of Occupational and Environmental Medicine.