480 likes | 605 Views
IMPACT EVALUATION OF PERFORMANCE BASED CONTRACTING FOR. A collaboration between the Ministry of Health, CNLS, SPH, INSP-Mexico and World Bank. GENERAL HEALTH AND HIV/AIDS SERVICES IN RWANDA. Presentation plan. Country Profile Impact evaluation design
E N D
IMPACT EVALUATION OF PERFORMANCE BASED CONTRACTING FOR A collaboration between the Ministry of Health, CNLS, SPH, INSP-Mexico and World Bank GENERAL HEALTH AND HIV/AIDS SERVICES IN RWANDA
Presentation plan Country Profile Impact evaluation design Results from the general health baseline and Follow up Results from HIV baseline and Follow up studies Discussion 2 9/1/2014
Study Rationale No examples of rigorously evaluated bonus payment schemes to public sector health care providers in developing countries No distinction between the incentive effect and the effect of an increase in resources for the health facilities Link between worker motivation programs and quality of care 8 9/1/2014
TRADITIONAL FINANCING VS PBF Traditional Way of Health financing in Rwanda was based on 3 sources of finance: GoR / Founders: Investment cost (Infrastructures & Equipment) GoR / Private / Population: Recurrent cost (salaries, drugs, materials, trainings,..) PBF: funds received for remuneration can be derived for enhancing human capacity building, HF materials, or any new strategy aiming at increasing quality.
Hypotheses For both general health services and HIV/AIDS services, we will test whether PBC: Increases the quantity of contracted health services delivered Improves the quality of contracted health services provided Does not decrease the quantity or quality of non-contracted services provided, Decreases average household out-of-pocket expenditures per service delivered Improves the health status of the population 10 9/1/2014
Evaluation Design Make use of expansion of PBC schemes over time The rollout takes place at the District level Treatment and control facilities were allocated as follows: Identify districts without PBC in health centers in 2005 Group the districts in “similar sets” based on characteristics: rainfall population density livelihoods Flip a coin to assign districts within each “similar” to treatment and control groups. 11 9/1/2014
Roll-out plan Phase 0 districts (white) are those districts in which PBF was piloted Cyangugu = Nyamasheke + Rusizi districts Butare = Huye + Gisagara districts BTC = Rulindo + Muhanga + Ruhango + Bugesera + Kigali ville Phase 1 districts(yellow) are districts in which PBF is being implemented in 2006, following the ‘roll-out plan’ Phase 2 districts (green) are districts in which PBF is not yet phased in; these are the so-called ‘Phase 2’ or ‘control districts’ following the roll-out plan. According to plan, PBF will be introduced in these districts by 2008. 13 9/1/2014
Program Implementation Timeline 16 9/1/2014
More money vs. More incentives Incentive based payments increase the total amount of money available for health center, which can also affect services Phase II area receive equivalent amounts of transfers average of what Phase I receives Not linked to production of services Money to be allocated by the health center Preliminary finding: most of it goes to salaries
The baseline has 4 surveys General Health facility survey (166 centers) General Health household survey (2,016 HH) HIV/AIDS facility survey (64 centers) HIV/AIDS household survey (1994 HH)
Baseline Field Sampling: GH HH • Randomly selected FOUR CELLS per HF • Done in SPH using Epi Info and given to team leaders • Randomly selected THREE ZONES per cell • Done by GH HH team leader in field • Obtained household lists for each of the zones and randomly selected ONE HOUSEHOLD per zone • Done by GH HH team leader in field with zone/village leader • Random sample of 12-13 households with children < 6 years old per HF • 2,159 households.
Validity of Sample • Require two different validations: • Validate the sampling for the evaluation design • Diff in means tests between Phase I and Phase II to determine if intervention and comparison groups balanced at baseline • Validate the quality of data • Compare descriptive stats to other sources of national data (i.e.: 2005 & 2007 DHS, MOH data)
Analysis Plan All analyses will be clustered at the district level Compare the average outcomes of facilities and individuals in the treatment group to those in the control group 24 months after the intervention began. Use of multivariate regression (or non-parametric matching) : control confounding factors Test for differential individual impacts by: Gender, poverty level Parental background (If infant : maternal education, HH wealth) 23 9/1/2014
Difference in differences models To test the robustness of the analysis Control sample (both observed and unobserved) heterogeneity A two-way fixed effect linear regression: 24 9/1/2014
General Health Centers Survey:Content All 166 centers in General Health General characteristics Human resources module: Skills, experience and motivations of the staff Services and pricing Equipment and resources Vignettes: Pre-natal care, child care, adult care VCT, PMTCT, AIDS detection services Exit interviews: Pre-natal care, child care, adult care, VCT, PMTCT
Baseline Health Facility:Human Resources On average: 1 doctor for every 31,190 individuals, 1 nurse for every 4,835 individuals 30
Baseline Household Data: Child Recent Illness and Symptoms (<6 years)
Validity of Sample and Data 46 • Evaluation design • Of 110 key characteristics and output variables of HF, the sample is balanced on 104 of the indicators. • Of 80 key HH output variables, the sample is balanced on 73 of the variables. • Quality of data • HH Results comparable to the 2005 DHS, MOH data
Follow-up Field Sampling: GH HH • Result: • In total 2159 were suppose to be surveyed in the catchment area of 167 facilities. • 1888 (87%) were interviewed in 2006 and 2008 • 267 (12%) were replaced • 4 (0.2%) were not found and not replaced • % of replacement by region: • South (24%), North (14%), East (4%) and West (13 %)
Baseline Household Sample • 38 ARV facilities for household sampling • 13 MAP; 10 Phase I; 15 Phase II ARV sites for HH sampling • Original sample of 1,487 patients from health facilities and associations • Final sample consists of 1,961 households and 7,494 individuals
Household Level: Sample Size of Patients • We find there is an additional sample of individuals in our household sample who self-report HIV+ status • Based on this, final distribution of patients:
Patients vs. Non-Patients:Mortality in the Household in the last 5 years
Follow-up Field Sampling: HIV HH • Objective: • Return to the same households to create panel data set (2006-2008) • Print baseline roster (names, codes) and keep consistent across waves • Account for household members who left and new arrivals from 2006-2008
Follow-up Field Sampling: HIV HH • Result: • In total 1996 were to be surveyed in the catchment area of 38 ARV facilities. • 1,716 (87.06%) were interviewed in 2006 and 2008 • 280 (12.94%) were not found and not replaced • % of missing by region: • South (12.6%), North (12.7%), East (7%) and West (16.4%)