1 / 26

Measuring Outcomes in EAP and Workplace Programs: Intro of a new Suite

Measuring Outcomes in EAP and Workplace Programs: Intro of a new Suite. David Sharar, PhD Managing Director Chestnut Global Partners. NBC EAP Group Meeting San Diego September 9, 2009. Richard Lennox, PhD VP of Commercial Science Chestnut Global Partners.

shiloh
Download Presentation

Measuring Outcomes in EAP and Workplace Programs: Intro of a new Suite

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Outcomes in EAP and Workplace Programs: Intro of a new Suite David Sharar, PhD Managing Director Chestnut Global Partners NBC EAP Group Meeting San Diego September 9, 2009 Richard Lennox, PhD VP of Commercial Science Chestnut Global Partners

  2. Performance measurement in EAP is more like a Rorschach test than a defined term.

  3. "I need to move beyond 'counting noses' and begin to quantify the actual impact of EAP intervention to ensure our continued viability".Michael Hack, LCSWManager, Global Workplace Solutions - ConocoPhillips, Houston, Texas

  4. Performance is in the Eye of the Beholder • Purchaser: Are my EAP dollars being spent wisely? • Provider: (primary vendor or affiliate): Does good service or results produce rewards? • Client (employee or family member): Is the EAP responsive to my personal needs?

  5. State of performance measurement in EAP • Substandard performance is largely invisible • Many measures are blunt, incomplete, distorting, exaggerated (e.g. Utilization rates) • We lack common definitions & standard markers of success • Under capitation, “marginal” performance receives the same rate as “optimal”

  6. Which results in: • Purchasers use the one metric everyone knows: PRICE • Purchasers and their agents lack understanding of what an EAP should accomplish • The link between Outcomes and Price is unclear (and not known) • So, employers buy EAP on Price and Promises

  7. Why measure Outcomes? • Describe the effects or results of our interventions (outcome measures) • Improve an aspect of the process of care (process measure), which in theory leads to better outcomes • Make comparisons across vendors or program models • Counterforce to deflation of EAP rates

  8. Issues with “Employer-Driven” • Many employers don’t know a good measure from a bad one • Lack of senior management engagement • Over-reliance on consultants and brokers who do not understand EAP • Competing goals between finance, HR, benefits, occupational medicine, etc. • Performance measurement is difficult-and this difficulty is not always appreciated

  9. Why are outcomes so difficult? Natural Problems • Confounding factors beyond your control • Sample size too small to produce effect • Long delays when measuring over time • Low frequency of interesting outcomes

  10. Why are outcomes so difficult? Human Problems • Inadequate information systems • No extra funding (vendor bears cost) • Accessing employer data • Point of measurement complexity • Insufficient level of clinical detail • How does one address poor outcomes?

  11. Criteria for selecting measures: • Does the measure serve to enhance the productivity or well-being of employees? (e.g. is it “mission critical”? • Is the measure based on science or opinion? (and if opinion, is there consensus)? • Is it feasible? (resource availability, automated data collection, and statistically meaningful comparisons)

  12. Outcome Variables • Defined • Occur after the intervention • Caused by the intervention • Not by some other variable • Is closely linked to the intervention • Not some spurious factor • Affect by other factors

  13. Multiple Determinism “The characteristic of a variable to be affected by a large number of factors in addition to the condition in question, increasing the risk that a failure to changed the variable is incorrectly attributed to a weak intervention rather than to the fact that the outcome is a poor indicator of effectiveness.”

  14. Proximal Outcomes • Variable central to intervention • Linked to the specific goals of the intervention • Occur at the end of EAP intervention or discharge from treatment

  15. Medial Outcomes • Quality of Life • Impact of the intervention on the client’s life experience • Not directly related to the goals of intervention, but not too far removed • In the real world • At work, or home • ADL (activities of daily living) • Also affected by other illnesses or disorders

  16. Distal Outcomes • Only indirectly related to the goal of intervention • Health Care use – clients is not told to avoid using medical care • Caused by numerous other factors

  17. Clinical Treatment Evaluation • Proximal Outcomes • Most important for testing efficacy • Provides the most convincing test of whether intervention works as planned • Will have the strongest effects • Best for establishing sample size • Best for establishing the ability to move clinical markers.

  18. Effectiveness Studies • Designed to evaluate ability of EAP to affect post-clinical variables. • Best suited for medial outcomes • Quality of life: Does the intervention improve the clients life beyond simply reducing distress? • Can the client function at work?

  19. EAP Evaluation • Not focused on efficacy • Interested in the real world at work • Can make better use of distal outcomes • Economic analysis can be linked • Smaller effects • Large sample • Secondary data

  20. Take Home Message • Understand the proximity of your outcome to the intervention its self. • Incorporate multiple determinism of an outcome in the expected effect size and the statistical power. • Don’t burden you intervention by trying to show extraneous value that will be difficult to detect in any efficacy study

  21. CGP's New Suite of EAP Outcome Measures • SHORT but empirically valid • Easy to administer • Workplace focused (5 subscales, 5 items per subscale) • Norms with non-clinical population being established for comparisons • FREE of charge (but we would like the data)

  22. Absenteeism Response Key: 1= strongly disagree; 2= somewhat disagree; 3 = neutral; 4 = somewhat agree; 5 = strongly agree

  23. Presenteeism Response Key: 1= strongly disagree; 2= somewhat disagree; 3 = neutral; 4 = somewhat agree; 5 = strongly agree

  24. Work Engagement Response Key: 1= strongly disagree; 2= somewhat disagree; 3 = neutral; 4 = somewhat agree; 5 = strongly agree

  25. Life Satisfaction Response Key: 1= strongly disagree; 2= somewhat disagree; 3 = neutral; 4 = somewhat agree; 5 = strongly agree

  26. Workplace Distress Response Key: 1= strongly disagree; 2= somewhat disagree; 3 = neutral; 4 = somewhat agree; 5 = strongly agree

More Related