Jim Bellows, PhD MPH Putting Care at the Center Conference November 16, 2017

1. Evaluating Social and Economic Interventions in Complex Populations –How Do We Learn Better and Faster? Jim Bellows, PhD MPH Putting Care at the Center Conference November 16, 2017

2. Evaluating social interventions in complex populations –Barriers to address; progress made and pending Biggest Barriers Understanding the contribution of social components in multi-dimensional programs • Study designs that are sufficient to yield robust evaluation of multi-dimensional complex care programs • Study populations large enough that the subpopulations who receive social interventions provide sufficient power for evaluation • Structured, reliable data about individual participants social needs, interventions delivered, and related outcomes Aggregating and synthesizing results across programs • Alignment on study populations (or subpopulations) and outcomes Long-term commitment and resources

3. Learning challenges in complex needs • Complex People • “Happy families are all alike; every unhappy family is unhappy in its own way.” • Virtually infinite array of unmet bio·psycho·social needs • Complex Programs • Members with complex needs encounter an array of programs, embedded in complex health care and community resource systems • Time • Hard to apply “rapid cycle learning” when outcomes develop over years • (Current Chaotic) Learning Environment • Lack of agreement on population or subpopulations – definition or identification • Weak, generic, and inconsistent outcome measures • Searching for a “pot of gold” rather than systematic, incremental learning • Inability to synthesize evidence across studies To do: move from chaotic to complex to complicated (someday to “simple”?) 3

4. An identification problem Given… • Program 1: IA (Px)  {O1, O2} • Program 2: IB+S (Py)  {O3, O4, O5} • Suppose: O3, O4, O5 > O1, O2 Questions: • Which program was more effective, IA or IB+S? • What was the contribution of S, the social intervention? Problem

5. KP’s path to stronger learning about complex care Recent progress • Agree on set of enhanced measures – outcomes, processes, and intermediate outcomes • Insist on appropriate evaluation methods – randomized or quasi-experimental designs, intention-to-treat analysis, etc. • Replicate (closely!) to understand reproducibility and generalizability • Get power via adequate size Future Work • Align on study populations and patient assessment tools • Track specific needs identified and interventions delivered, within “complex care” programs

6. Recommended measures (measures in bold are most important) DRAFT

7. Tight replication across sites Objectives • Assess reproducibility • Increase total sample size to increase power and enable study of subpopulations “Primary Care Plus” • Development site found reduced inpatient costs and high patient and provider satisfaction, in a matched cohort study • Now – two replication sites, in another KP operating Region • Tight fidelity to original model was as a leadership precondition • clinical team • population identification criteria • patient assessment instruments, work flows • EHR documentation • outcome measures PLUS added a randomized evaluation design

8. KP’s path to tracking social needs and interventions A fully functional system for social documenting social needs provides the data to assess the specific role of social interventions in complex populations KP has invested heavily in our EHR; could we pair it with a “social EHR”? Our cutting edge, developed in one KP Region, a data system with functionality analogous to EHR, but focused on social needs… • “Test results”  needs identified on social needs screening • “Diagnoses”  unmet needs assessed and confirmed • “Orders”  community resource referrals made • “Procedures”  community resources connections made … for any patient (complex or not) in any setting

9. Evaluating social interventions in complex populations –Barriers to address; progress made and pending Biggest Barriers Understanding the contribution of social components in multi-dimensional programs • Study designs that are sufficient to yield robust evaluation of multi-dimensional complex care programs • Study populations large enough that the subpopulations who receive social interventions provide sufficient power for evaluation • Structured, reliable data about individual participants social needs, interventions delivered, and related outcomes Aggregating and synthesizing results across programs • Alignment on study populations (or subpopulations) and outcomes Long-term commitment and resources

10. Long-term commitment and resources • Identified KP systemwide leadership for Complex Needs, made up of KP’s highest-level clinical/quality leaders First action – a new direction: “Develop and implement a systematic, cross-Regional plan to design, test, and scale programs addressing high cost, high need patients. For existing models strengthen design and evaluation elements where possible. …” Subsequent actions have driven alignment with this approach • Sponsored process to identify and prioritize research questions related to social needs and social interventions • Launched new multi-investigator, multi-Region research networks for both social needs (SONNET) and complex needs (CORAL) …with collaboration between them and KP operating funds to support

11. Discussion Your thoughts? Jim Bellows, PhD MPHManaging Director, Evaluation and AnalyticsKP Care Management InstituteJim.Bellows@kp.org 11