Quality and Accountability in Health: Audit Evidence from Primary Care Providers

1. Jishnu Das (World Bank and Centre for Policy Research) With AlakaHolla (World Bank) KarthikMuralidharan (UCSD) Quality and Accountability in Health: Audit Evidence from Primary Care Providers SITE June 2013

2. The Problem Strong theoretical reasons why health care should be public • U(government) ≠ U(Consumer): Pendergast (2003) • Patient satisfaction among narcotic addicted patients not a good measure of how good the doctor is • Private sector aggregator of customer feedback • Medical care arguably a credence good • You don’t know what you need, but observe utility from what you get • Widely believed to produce inefficiencies in the market • Darby and Karni (1973): Over-treatment • Wolinsky (1993): You can’t observe what you bought; treat “low”, charge “high” • Gruber and Owens (1996): Caesarian sections • Balfoutas and others (forthcoming): Greek taxi drivers (over provision and over charging)

3. And yet… • 80% of first-contacts (primary care) in India in private sector • New nationwide study: 77 percent of private providers in rural areas do not have medical training • Contrast: All public providers are (supposedly) trained, the majority with an MBBS 77% of providers have no degree, 18% have some other degree (BAMS, BIMS, BUMS, BHMS), and only 4% have an MBBS degree (roughly equivalent to MD in the U.S.). Average village has 3.36 providers with no degree, 0.80 providers with some degree, and 0.18 providers with an MBBS degree

4. And yet… • Not (just) because there aren’t enough public sector providers Public share increases from 20% to 35% in villages where there is a public doctor but households still visit private providers in 65% of primary care cases.

5. Why? • What people demand from health care providers very different from what the public sector provides • Hypothesis 1: Decreases quality, increases costs • Example: Demand for injections/steroids leads to lower quality for higher cost • Peer and Administrative accountability from experts in regulated (and in low-income settings) health care beats customer accountability through the market • Hypothesis 2: Increases cost, but at increased quality • Example: Poor governance in public sector (Chaudhury and Hammer 2004, Chaudhuryet. al. 2006, Das and Hammer 2007) • Low effort arising from poor administrative accountability is hard to quantify • But is potential one large source of losses: `Quiet Corruption’

6. In health care: Arrow (1963)

7. This paper • Use audit studies with patient and provider fixed-effects to assess quality in public and private sector • 22 people recruited from the local community and extensively trained visits multiple providers presenting the same set of symptoms. Providers do not know that this is not a real patient • Show effect of practicing in private sector on • Adherence to medically required checklists • Under-treatment • Over-treatment • Assess whether there is a price-quality relationship in private sector

8. Three literatures it relates to • Customer Accountability in private sector versus administrative accountability in public sector • What’s out there and what we don’t know • New empirical and theoretical literature on credence goods • Dulleck and Kerschbamer (2006, 2011, 2012) • Schneider (2012), Balfoutas(forthcoming) • Bonroy and others (2012) • Audit studies in labor markets and services • Primary around issues of discrimination • What are we adding

9. This paper: What • Deploy “standardized patients” (audit study) • People recruited from local communities and extensively trained to present with the same symptoms to multiple providers • Largest such study to date (1105 interactions) • 2 Related studies • Compare market care to provider in public clinic (64% not doctors) • Compare same doctor in public and private practice • Note: unlike audit studies of car buying or home rentals, we always observe a completed sale • Problems arising from potentially off-equilibrium behavior may still remain • We try out various strategies to interpret these results in the light of known issues with audits

10. Overview of Results • Significant evidence of over-treatment and under-treatment relative to medical protocols • Conditions are not diagnosed or treated appropriately • Many medicines are not required • BUT • Public clinics provide similar care to private clinics • Effects vary depending on measure of quality used • Joint effect of public sector with provider characteristics • 72% private sector providers have no medical training • The same provider in his/her private clinic provides better care than in his/her public clinic across all quality measures • Customer accountability rewards better quality with higher prices • No link between provider wages in public sector and measures of quality

11. Remainder of talk • Where we worked (and what does it look like) • What we did • What we found • Ruling out (some) interpretations of the data • Worry in particular about off-equilibrium behavior

12. This paper: Where? • All districts divided into 5 Socio-Cultural Regions (SCRs); one district from each SCR • 20 randomly chosen villages from each district • Representative sample of all types of providers in 3 districts of Madhya Pradesh (and public providers in 2 more); majority has no medical training • Additional sample from (urban) Delhi

13. MP Study: The sample

14. Rural India: MP • 100 villages in MP, randomly selected in 5 districts—located >1000 health care providers • Snapshots of the two remotest districts

15. Standardized patientssample • Sample restrictions • Ruled out 2 remote districts entirely for private market • Ruled out remote locations in other 3 districts • Sampled • All MBBS private providers • All public clinics in all districts • But no more than 2 doctors per clinic • All private clinics of public doctors in all districts • Add in untrained till we have 6 providers per sampled village

16. Basic Sample Description

18. Standardized patients • SPs • 22 SPs recruited from the local community • Important so that their appearance and manner conform closely to providers’ expectations • Thoroughly trained to make plausible excuses to avoid invasive exams • “palm” medicines if required • 150+ hours of training • First tried in Delhi pilot • No adverse events; <1% detection rate

19. Standardized patients • Three standardized cases • Unstable Agina: “Doctor, this morning I had a pain in my chest” – Ramlal, Male, 45 years old • Proxy Dysentery: “Doctor, my 2 year old child has been suffering from diarrhea for 2 days” – Shankarlal, Male, 25 years old • Asthma: “Doctor, last night I had a lot of difficulty in breathing” – Rajesh (Male) or Radha (Female), 25 years old • Cases chosen such that • Relevance to the Indian context • Increasing incidence of cardiovascular and respiratory illness in India • Diarrheal diseases kill approximately 200,000 children per year (Black et al. 2008) • No invasive treatment required • Important to minimize any potential harm to SPs

20. Standardized patients • What is measured • Quality of care through adherence to required and essential checklist of questions and examinations that the provider should complete for each patient • Why this may be preferable • Treatment: correctness, incorrectness, use of antibiotics and steroids for cases where they are not required • Diagnosis: whether given, whether correct • Time spent, total questions asked, total examinations completed

21. Relation between quality measures • Doctors under-treat because they figured out that these were not “real patients”. But then, we should see that “correct treatment” is less likely for doctors who spend more time and complete more of the checklist, since they would be more likely to figure out that the patient is not “real”. We find exactly the opposite • Little evidence of signaling through medically irrelevant costly effort: more effort leads to better treatment through 90 percent of the distribution

22. Basic Sample Description

23. Results • Checklist adherence • IRT Scores • Treatment • Diagnosis • Prices

24. Checklist adherence

25. Checklist adherence

26. Checklist adherence

29. IRT Score

33. What exactly is happening with treatment? • In audit 1, the two groups behave similarly, but there are some differences across cases • MI identical in both • Dysentery public sector providers 20-30% more likely to ask to see child, no difference in ORS • Asthma public sector providers 12-14% less likely to give correct treatment (not statistically significant) • Across all cases, public 13% less likely to give antibiotics • Dual sample, with and without provider fixed effects • MI: Equal likelihood of EKG/Referral but private more likely to give Aspirin • Dysentery: Public 10-12% (not significant) more likely to give ORS, private 18-20% more likely to ask to see child • Asthma: Public 13-15% less likely to get it correct

36. Diagnosis • Problem: 67% interactions there is no diagnosis • Noted in pilot • Final survey: randomized SSPs into 2 groups • 1 group turns around as they are leaving and ask the provider “Doctor, what is wrong with me?” • Increases rate of diagnosis provision by 20-25 p.p. in all groups • First look at likelihood of providing diagnosis • Second, use randomization as instrument in selection model with binary dependent variables to deduce correct diagnosis rates

41. Prices in the Private Sector • Huge variation within providers • True for all MBBS, other qualified, and unqualified providers Each vertical line represents a box-plot of prices charged by a provider to real patients. Providers are sorted on the x-axis by quality (measured by number of questions asked and examinations conducted)

42. Prices, Checklists and Treatment • Greater compliance with the checklist is always rewarded in higher prices • Correct treatment leads to higher prices , but vanishes once we control for checklist adherence • Stronger premium among MBBS providers • Works across all cases, weaker for asthma

43. Prices and Checklist Adherence

47. Does the public sector reward quality? • Public sector pay in India follows a matrix • Composed of: rank, tenure, qualifications • Zero effect of checklist adherence, treatment, likelihood of discussing diagnosis on wages

50. Quick back of the envelope • We can provide a back of the envelope measure of costs and quality in the public and private sectors • This is rough—public and private sector providers provide other services beyond primary care