Less Pain, More Gain : An Evidence-Based Approach to Long-term Deficit Reduction

1. Less Pain, More Gain:An Evidence-Based Approach to Long-term Deficit Reduction Jon Baron Coalition for Evidence-Based Policy March 2013

2. Coalition for Evidence-Based Policy A nonprofit, nonpartisan organization. Mission: To increase government effectiveness through rigorous evidence about “what works.” Coalition has no affiliation with any programs or program models – thus serves as an objective, independent resource on evidence-based programs. Funded independently, by national philanthropic foundations (e.g., MacArthur, William T. Grant).

3. Program reforms shown to produce major savings without loss of quality or benefit: Successful welfare-to-work programs(Riverside, L.A., Portland) • Evaluated in large, convincing randomized trials • Increased employment and earnings by 20-50% • Produced net taxpayer savings of $1700-$6000/person • Had important policy impact (1996 welfare reform act)

4. Cost-saving program reforms, continued: Transitional Care Model (a nurse-led hospital discharge and home follow-up program) • Evaluated in two well-conducted randomized trials • Reduced rehospitalizations 30-50% • Produced net savings of $4000/patient • Successful national replication could save $10 billion/year

5. Cost-saving program reforms, continued: Subsidized Guardianship as alternative to foster care • Evaluated in large randomized trial in Illinois • Increased permanent placements by 8% • Reduced days in foster care by 16% • Saved Illinois $2300/child ($90 million over 10 years) • Expanded nationally (2008), CBO scored $800M savings

6. Too few examples currently exist, because: • Rigorous evaluations are still relatively uncommon in most program areas. • Promising ideas, when rigorously tested, too often are found not to work.

7. Promising ideas, when rigorously tested, too often are found not to work Medicare Coordinated Care Demonstration (MCCD) • Preliminary studies had suggested large cost savings (e.g., 25%) • However, the more definitive, randomized MCCD found a net cost increase of 11%.

8. Other examples: In medicine: 50-80% of interventions found promising in phase II (nonrandomized studies or small efficacy trials) are found ineffective in phase III (sizable RCTs). In business: Google and Microsoft have conducted ~13,000 RCTs of new products/strategies, 80-90% of which found no significant effect (Manzi). In K-12 education: Of 120 interventions evaluated in large, convincing RCTs funded by Institute of Education Sciences 2003-2012, ~90% had weak or no effects.

9. Proposed strategy for long-term deficit reduction: “If you want to increase your success, double your failure rate.” - Thomas Watson, IBM • Accelerate development of promising, innovative program reforms with potential for cost savings. • Greatly increase the number that are rigorously evaluated.

10. How to do this? • Low-cost rigorous evaluations make it possible to test hundreds of program models/strategies each year, rather than a select few (as is currently done). • Federal “waivers” from law/regulation can be used to stimulate state and local innovation and evidence-building aimed at entitlement savings. • New high-visibility “prize competition” seeks to jump-start low-cost trials and testing of cost-saving reforms.

11. Coalition for Evidence-Based Policy www.coalition4evidence.org Jon Baron jbaron@coalition4evidence.org 202-683-8049

12. Examples: Medicare Health Support Pilot (disease management) • Preliminary studies suggested a high ROI (3:1 to 7:1) • However, large randomized MHS found a net cost increase of 4-11%.

13. OMB & CBO Budget Scoring Creates New Incentive for Development/Expansion of Evidence-Based Programs • OMBwill credit entitlement savings a program generates, that have been demonstrated in “rigorous experimental research.” (2010 guidance) • CBO stated that it “gives greater weight to demonstrations and experiments that use random assignment …” when scoring the cost of legislation. (2011)

14. Comprehensive Child Development Program: Impact on Mothers’ Employment Rate

15. Impact on Family Welfare Receipt

16. Impact on Percent of Children “At Risk” in Cognitive Development & Behavior Treatment

17. Impact on Percent of Children “At Risk” in Cognitive Development & Behavior Treatment Control

18. Impact of Career Academies on Completion of a Postsecondary Credential Randomized Trial Results Comparison Group Study Results* *The comparison group consists of similar students in similar schools nationwide. Their estimated rates of postsecondary completion are statistically adjusted to control for observed differences between their background characteristics and those of the Career Academy group. Career Academy Students Non-randomized Comparison Group* Randomized Control Group Career Academy Students Source: Data provided by James Kemple, MDRC Inc.

19. Websites that identify evidence-based programs Top Tier Evidence www.toptierevidence.org Social Programs that Workwww.evidencebasedprograms.org Blueprints for Violence Preventionwww.colorado.edu/cspv/blueprints/index.html What Works Clearinghouse (K-12 Education)www.ies.ed.gov/ncee/wwc Best Evidence Encyclopedia (K-12 Education)www.bestevidence.org