460 likes | 605 Views
Open Forum Milwaukee, Wisconsin June 12 th , 2013. Karl Ensign, Director of Evaluation Association of State and Territorial Health Officials (ASTHO). Estimating Return on Investment for Public Health Improvements Introduction to a new t ool. Helps answer the following questions:
E N D
Open Forum Milwaukee, Wisconsin June 12th, 2013 Karl Ensign, Director of Evaluation Association of State and Territorial Health Officials (ASTHO) Estimating Return on Investment for Public Health ImprovementsIntroduction to a new tool
Helps answer the following questions: • Are we making the right investments? • Are we becoming more efficient? • What bang are we getting for our buck? • What is our budget accomplishing? • Are we being good stewards? • Must be done thoughtfully and carefully ROI is one way of measuring and communicating public health effectiveness in a manner that is particularly salient for policymakers, funders, administrators and the general public
ROI is a Specialized Form of Analysis • Outcomes are monetized: • Compares the costs of an intervention with its benefits – in financial terms • Yields the net return on investment – over time • Sensitivity analysis can be conducted – different levels of investments and their benefits
ROI Calculated: • Net benefit = Benefits – Costs • ROI = Benefits – Costs Costs • Hypothetical values: • $5 = ($400 + $500 + $300) – ($150 + $50) ($150 + $50) • Or …
“…a dollar spent on pediatric immunization is estimated to save $5 in treating preventable illness.”
Where and How has ROI been used in Public Health? • Public health programs (injury prevention, tobacco prevention and control) • Aggregate public health spending (Glen P. Mays, UKY) • Improvement projects undertaken by agencies
Challenges from an Agency Perspective • New data probably need to be collected • An investment may take years to produce benefits • Benefits may accrue outside the agency (population health) • Benefits may be difficult to link back to a specific public health program(s) or function(s)
ROI of QI – Advantages and Disadvantages • ROI is more immediate • Timeframe is shorter • ROI accrues more directly to the agency • Tends to be narrower in scope • Return is more modest
Hot off the PressesTHE NEW ROI TOOL!! • Funding from Prevention & Public Health Fund, ACA • Measures ROI for improvement projects • QI projects undertaken through the National Public Health Improvement Initiative (NPHII) • ASTHO led workgroup provided input and guidance • CDC, state and local agencies, foundations, academia • Developed by Glen Mays, University of Kentucky • Beta tested by Connecticut, Maine, and Virginia
Pathways to Realizing ROI for QI • Reductions in standard operating costs • Greater efficiencies realized • Revenue enhancements • Increased cost reimbursement • Increased productivity of agency functions • Increased service encounters • Decreased time to produce outputs • Reduced cycle time process
How the Tool Calculates ROI • ROI = Benefits – Costs Costs Improvements in Routine Operations – Investment Costs Investment Costs (Routine Operations + Other Outcomes) – Investment Costs Investment Costs
Incorporates Standard Accounting Practices in ROI Calculation • Amortization • The cost of an investment should not be absorbed entirely in the first year • Amortization rate spreads the agency’s cost/investment over the useful life of the product • Present value • The relative worth of a single dollar changes over time • Accurate comparisons are made by applying a discount rate (inflation) to • Costs • Returns
Tool Can Be Used throughout Project • Prospectively – Planning Phase • Implementation Phase • Retrospectively – Post Implementation
Apply ROI Correctly • Build evaluation methods – including ROI – into program inception • Clearly specify intended purpose and use of ROI
Conduct ROI through a transparent process • Conduct ROI through an inclusive process • Implementers and end users of the analysis
Testing the New ROI Tool in Connecticut: A Prospective Look Susan Logan, MS, MPH Connecticut Department of Public Health June 10, 2013
Improving the Connecticut Department of Public Health Databases • AIM: Make three DPH databases compliant with the CT DPH policy on collecting sociodemographic data (similar to federal OMB-15 directive in 1997) • Increase the percentage of compliant databases from 4% to 10% • Put a plan in place to modify the remaining databases • Make sure that all newly-designed databases are aligned with the DPH data collection policy. • The modification process for the targeted databases was implemented and this strategy will be followed on a continuous quality improvement basis through 2014. Background on Quality Improvement Initiative
Determining Investment Costs: QI Planning and Implementation • Pre-implementation/Baseline Period • Project team mid-point annual salaries x FTE spent (plus Fringe) • Include employees assisting with planning • Office operations costs (printing, supplies) • Travel for grant award • Implementation and Post-Implementation Periods • Project team mid-point • annual salaries x FTE spent (plus Fringe) • Include employees assisting with implementation • Cost of contracting out services • Office operations costs (printing) • Travel for grant award • Modifications to Databases • Communications (storyboards, easels, DPH Newsletter)
Approximately how many meetings did you attend regarding this QI project? ________ • Approximately how much time did you spend on the following activities related to this QI project? • Attending formal or informal meetings? hours • Making changes to data collection forms? hours • Making changes to databases? hours • Generating cost estimates for changes? hours • Other activities related to this QI project? hours • Please briefly describe these activities: ________________________________________________________________________________________________________________________________________ Customer Satisfaction Survey on QI Process Collected Data on Time Spent on Planning and Implementation Activities:
Takes into account time prds (4.7mos baseline) Salary Costs for QI Initiative Team
Salary Costs for QI Initiative Team: Implementation Prds 2 and 3 Data quality committee reviewing databases and overseeing fixes (ImplPrds 2 - 3)
Non-Personnel Investment Costs: Planning and Implementation Periods
Determining Costs and Savings for Outcomes and Outputs: ROI Produced • Baseline and Post Periods • Measures of Production Time • Time to analyze data with standardized sociodemographic categories (expected reduction) • Time to review databases (improves over time) • Improve Health Outcomes: Based on finding new cases in sociodemographic groups • Taking into account the Investment and Routine Operating Costs Only • Implementation Period 1 • Benefit (savings) was 64 cents for every dollar spent • Implementation Period 2 • Cost was $2.21 for every dollar spent • Adding in Outcomes • Implementation Period 1 and Overall • Savings was $17.05 ($18.16 overall) for every dollar spent
Did not include data in ROI calculations (unchecked) Outcome/Output Measures: Reach Outcomes: Rates Used these rows as a worksheet for the health outcomes on next slide
Include data in ROI calculations (checked) Outcome/Output Measures: Health Outcomes Savings
Used prospectively in this case • Seemed daunting at first, but once data is collected, calculations are done for you • Used customer satisfaction survey to gather data on time spent on planning and making modifications to databases • Customized to fit our needs Highlights of Using the ROI Tool for a Connecticut DPH QI Project
Can be applied for many different types of QI projects • Data quality projects may not have a large return, but it is the “right thing to do” • Projects that improve efficiency and reduce process steps/time will see a greater return on investment • Future Use of the ROI Tool: • ASTHO National Demonstration Project (3/1/12 – 10/31/13) • Make data accessible to local health departments – reduce time for state DPH staff to handle data requests • Save time cleaning up data Applications of the ROI Tool
Determining Return on Investment (ROI) Josh Czarda, Performance Improvement Manager, Virginia Department of Health
Example 1: Justifying Continuing Operations Increase Enrollment in Plan First – A Brief Background (A Medicaid Family Planning Program) Knowledge Individuals Providers Enrollment Process Eligibility Application Process/ Form Length Registration WebVision DSS System Understanding & Function Birth Control Knowledge Demographic, Age, Income, Insurance No More Needed Eligible LHD STD Testing Possible - Prompt for Plan First. Clerk still must know parameters Service Eligibility Levels Knowledge Other – Family Planning Ask Questions to Determine Eligibility Title X & Payment Issues Service Provided Auto-Enrollment & Payment External Provider If eligible – fill out application/ self populates some fields WIC Clinic Billing Not Interested Application Given to Individual If Interested Individual Signs & Dates Tracking/ Notification LHD Bills Medicaid Post Partum Tracking & Follow Up Capability No Incentive LHD Submits Printed App to DSS Individual Follows Up
Background - Implementing Changes and Greater Challenges Realizing Health Outcome ROI
Step 1: Calculating All Costs Pre-Implementation – Implementation Phases Initial PI Teams Costs DMAS, DSS & Local Health District Staff Additional Time, Billing & Enrolling Hired 2 FTEs to Run Program Marketing Outreach, Focus Groups, etc
Step 1: Calculating All Costs Pre-Implementation – Implementation Phases
Step 3. Measuring ROI of Long Term Health Outcome Impact • Working with DMAS to Determine Cost Per Pregnancy to Medicaid • Calculating Variation in Pregnancy Rate to Determine Savings • Calculating ROI Compares All Costs Including Medicaid Payments for Plan First
Example 2: Proactively Determining Is A PI Project Worth It? Automating the VDH Travel Process Traveler Number of TARs (1000) Estimated $130,000 Programming Costs Supervisor X Business Manager Time Spent Per Unit Office Director Optimistic Future State T&E $40,293 VS. X Executive Advisor Salary Cost Per Unit Deputy Commissioner = Commissioner Travel Cost Increase From Approval Delay = 73K in Time and Effort + Secretary HHS
Lessons Learned • The tool helps you recognize unanticipated costs and can be great for both prospective and retrospective decision making • ROI Analyses Must be Dynamic • PI Projects vary in complexity and metrics • ROI Tools must be flexible • Amortization of project costs can be challenging as change processes fluctuate • Illustrating & Showing ROI is Critical to Drive & Promote Continuous PI • Measuring Health Outcome ROI Remains A Critical Challenge • ROI for public health remains un-quantified for Public Health • A National Standardized Cost Analyses Calculating Cost Per Incident/Unit should be established