Evaluating OSHA inspections for intended and unintended outcomes

1. Evaluating OSHA inspections for intended and unintended outcomes Mike Toffel Associate Professor Harvard Business School Liberty Mutual Research Institute for Safety August 2012

2. May 18, 2012

3. OSHA, a much-criticized agency • Too lenient! (?) • Inspections too seldom • Penalties too small • Lengthy process to adopt new regulations compromises worker safety • Too costly! (?) • Stifling job creation / job killer • Increasing labor costs • Eroding America's competitiveness

5. Challenges evaluating impact of OSHA inspections:Causality • Most OSHA inspections are not random: • After accidents and deaths • When employees complain • If accidents/deaths are rare events, outcomes will feature mean reversion: • Problems likely decline after inspections… but even without inspections • Our approach • Examine random inspections and compare to a control group.

6. Challenges evaluating impact of OSHA inspections:Measuring outcomes • Several studies have relied on company logs • But inspections can lead companies to improve logs’ comprehensiveness, increasing reported injuries • This cloaks changes in actual injury rates • Our approach • Rely on workers’ compensation claims • Annual number of claims • Annual cost of all claims

7. 3 data sources OSHA IMIS Inspections Dun & Bradstreet Industry Single-plant Employment Sales • Data restrictions • California • Cal/OSHA • Some high hazard industries (randomization targets) • Single-plant firms WCIRB Number of claims Cost of claims Occupation classes Payroll

8. Data issues • WCIRB data • MOU between California Department of Insurance (CDI) and Commission on Health and Safety and Worker’s Compensation listing us as subcontractors • Required to keep data anonymous • OSHA data • Complex data structure • Limited documentation • Many conversations with Cal/OSHA to understand implementation of randomization process

9. Developing a matched sample • Treatments • Single-plant establishments randomly selected for a programmed inspection • High-hazard industries Cal/OSHA targeted for random inspection each year • Matched controls • Find population of single-plant establishments at risk of random inspection, but not selected • Exclude if < 10 employees or recently inspected • For each treatment, select one control: • Same industry and region • Closest size Result: 409 matched pairs

10. Matching led to a balanced sample of very similar treatments and controls • In the two years before the match year, the 409 matched pairs had very similar: • Sales • Employment • Payroll • Credit scores (PAYDEX, Comprehensive Credit Appraisal) • Annual number of WC claim • Annual total cost of WC claims

11. Figure S1: Indistinguishable levels

12. Figure S2: Indistinguishable trends

13. Industry distribution of matched sample (Table S2)

14. Evaluation model • Did treatments experience a greater decline in annual injuries (or injury-related costs) after inspections than the controls, examined over the same time period? • Fixed effects regression • Control for establishment characteristics • Difference-in-differences approach • Compares two groups over time

15. Randomized inspections reduce annual injuries by 9.4% Persistent

16. Randomized inspections reduce annual injuries by 9.4% Persistent

17. Randomized inspections reduce annual injuries by 9.4% Persistent effect

18. Randomized inspections reduce annual injury costs by 26% Persistent

19. Randomized inspections reduce annual injury costs by 26% Persistent

20. Randomized inspections reduce annual injury costs by 26% Persistent effect

21. Unanticipated consequences of inspections? • Inspections (and consequences) cause interruptions, but are they substantial? • Sales impact? • Credit worthiness? • Employment, payroll? • Firm survival?

22. Unanticipated consequences of inspections?

23. Unanticipated consequences of inspections? No difference in employment, payroll, sales (Table 2) • Tight confidence intervals enable us to rule out that inspections caused big declines of employment or payroll • No difference in credit ratings (Table S8) • Late bills, etc. more sensitive to financial burden than firm death • Two D&B metrics of financial distress: PAYDEX & CCA • No difference in firm survival (Table S7) • Approx. 5% of treatments and of controls died • Difference not statistically significant • Result robust to survival regressions

24. Summary of results • Evidence of intended results • Annual injuries reduced by 9.4% • Annual injury costs reduced by 26% • No evidence of unintended consequences • Sales impact • Credit worthiness • Employment, payroll • Firm survival

25. Conclusions • Cal/OSHA is not entirely toothless • Injuries and injury costs decline a lot • Social value of very approximately $350,000 per inspection • Includes medical costs and lost wages, but not productivity loss, pain and suffering, etc. • With many assumptions (similar effects across all states and inspection type, etc.) translates to $22 B per year nationwide • OSHA is not detectably a job killer • But we cannot rule out some costs Levine, David I., Michael W. Toffel, and Matthew S. Johnson. "Randomized Government Safety Inspections Reduce Worker Injuries with No Detectable Job Loss." Science 336, no. 6083 (May 18, 2012)

26. Future research in this domain 1. When do random inspections bolster regulatory compliance? • High vs. low hazardous industries • Strong vs. weak compliance history • VPP participants versus others • Prior OSHA consultation versus not 2. Do random inspections have spillover effects? • Organizational: bolster compliance at corporate siblings? • Geographic: bolster compliance at neighboring facilities? • Regulated domains: bolster compliance with EPA regulations? 3. Your ideas? • Loss prevention vs. OSHA inspections?

27. Other research… • Environmental reporting & carbon disclosure • Responding to public and private politics: Corporate disclosure of climate change strategies, Strategic Management Journal (2009) • Engaging supply chains in climate change. Working Paper (2012) • When do firms greenwash? Corporate visibility, civil society scrutiny, and environmental disclosure, Working Paper (2012) • Environmental ratings • How well do social ratings actually measure corporate social responsibility? Journal of Economics & Management Strategy (2009) • How firms respond to being rated, Strategic Management Journal (2010)

28. Environmental policy evaluation • Voluntary violations disclosure (EPA Audit Policy) • Coerced confessions: Self-policing in the shadow of the regulator, Journal of Law, Economics and Organization (2008) • Coming clean and cleaning up: Does voluntary self-reporting indicate effective self-policing, Journal of Law and Economics (2011) • Making self-regulation more than merely symbolic: The critical role of the legal environment, Administrative Science Quarterly (2010) • Mandatory performance disclosure • How firms respond to mandatory information disclosure, Strategic Management Journal (forthcoming) • Tailpipe emissions fraud • Competition and illicit quality, Working Paper (2012) • The role of organizational scope and governance in strengthening private monitoring, Working Paper (2012)

29. Quality management: ISO 9001 evaluation • Quality management and job quality: How the ISO 9001 standard for quality management systems affects employees and employers, Management Science (2010). • Occupational health and safety • Randomized government safety inspections reduce worker injuries with no detectable job loss, Science (2012).

30. Thank you! • Prof. Michael Toffel • Harvard Business School • Morgan Hall 497 • Boston, MA 02163 • mtoffel@hbs.edu • www.people.hbs.edu/mtoffel/