Measuring the Performance of Dental Clinics with Data Envelopment Analysis (DEA)

1. Measuring the Performance of Dental Clinics with Data Envelopment Analysis (DEA) To Advance Presentation Tap Space Bar or Left-click Mouse Darold T. Barnum, Ph.D., M.B.A. Professor of Management, and Professor of Information and Decision Sciences University of Illinois at Chicago 312-996-3073 dbarnum@uic.edu

2. Definitions • Productivity ≡ Outputs/Inputs • Outputs are measures of objectives • Inputs are measures of resources used • Suppose one objective is to fill caries, and one resource is dentist hours • Then, Productivity = • If dental clinics are not as productive as they could be, they provide less service than desired, or require taxpayers and patients to pay more than necessary Number of caries filledNumber of dentist hours

3. Measuring Dental Performance • Improving productivity is very important, but correctly measuring it can be very difficult • For example, suppose that the following inputs and outputs all are considered key

4. Illustrative Resource Inputs • Dentist hours • Hygienist hours • Assistant hours • Other labor hours • Facility and equipment value ($) • Materials expense ($) • Energy (KW hours)

5. Illustrative Physical Outputs • Emergency treatment (patient hours) • Caries filled (number) • Treatment for bleeding gums/bad breath/gum recession/tooth mobility (patient hours) • Denture repair (units) • All other procedures (units)

6. Illustrative Quality Outputs • Average waiting time for appointment (days) • Dental clinic/office hours per week • Average waiting time in office/clinic (minutes) • Average waiting time for emergency treatment (hours) • Patient Satisfaction Index (PSI)

7. Partial Productivity Indicators • We can compute 70 partial productivity measures by dividing each of the 10 outputs by each of the 7 inputs • Unfortunately, it is impossible to compare the overall productivity of clinics based on these partial indicators • To make overall comparisons between clinics, we need one comprehensive, all-inclusive measure with which to compare them

8. Comprehensive Measure • To get a single, comprehensive measure for each clinic, we could: • Multiply each output and each input by predetermined weights • Aggregate the weighted outputs, and the weighted inputs • Divide aggregated outputs by aggregated inputs However, it would be difficult to justify assigning equal weights to each input and output, or to defend some other weighting scheme.

9. Data Envelopment Analysis (DEA) • .Each clinic could argue, with considerable economic justification, that the heaviest weights should be assigned to those activities in which it excels • Another problem is how to set the criterion standard to which clinics will be compared, and how to determine the significance of variations from it • These issues are resolved by a methodology called Data Envelopment Analysis (DEA) • DEA uses linear programming to weight and aggregate inputs and outputs in a way that results in a single comprehensive productivity measure ranging from zero to 100 percent. • Productivity of a particular clinic is given as a percentage of the productivity of its most productive peers.

10. DEA (continued) • The criterion standard is automatically determined by the most productive clinics in the set of clinics being analyzed. • For each clinic, DEA sets the weights so the clinic will obtain the maximum productivity score when compared to its peers. • So, if a dental clinic scores less than 100%, this tells us that its peers are still more productive even when the weights are set to maximize the score of the given clinic. • A dental clinic is compared only to those of similar size and operating with similar input-output mixes. • If a dental clinic is less productive than its peers, DEA also identifies benchmark (best practice) peers that it should emulate to become more productive.

11. DEA (continued) • Once productivity levels have been identified, they can be analyzed to determine their causes. • This can be done qualitatively by doing comparative case studies on productive dental operations to determine why they are more productive. • It can be done quantitatively using the DEA scores as dependent variables and expected influences as independent variables in regression analysis.

12. Influences on Productivity • Suppose we want to estimate the impact on clinic productivity of such factors as • city size/density/mean income/region/other city characteristics • average case severity • procedures mix • patient age mix • English fluency level • social/cultural/economic factors (patients & staff) • broken appointment percentage • task distribution among dental occupations • size of operation • This knowledge could help identify the causes of low productivity • The estimates could be used to adjust clinic DEA scores for causes beyond their control

13. Stochastic Frontier Analysis • To obtain such estimates, we could regress the DEA scores on the independent variables of interest • Ordinary Least Squares or Tobit regression are inappropriate however • Just as clinics may be unproductive converting inputs to outputs, many may be unproductive in converting environmental factors into outputs • To account for this, we can use a statistical model that estimates both normal random fluctuations in the error term and downward biases due to low productivity. • Stochastic Frontier Analysis adjusts for low productivity in converting environmental conditions into outputs (v is random error, u is productivity shortfall) • If we could get data on a set of clinics in multiple time periods, we could use the very powerful methods of Panel Data Analysis, including trends or shocks

14. Example • Artificial Data for 30 dental clinics • Inputs • Labor hours • Value of equipment • Energy in KW hours • Outputs • Number of caries filled • Number of other procedures completed • Patient satisfaction index • Hours of patient consultation

17. After Obtaining the DEA Scores: • Do case studies of the best-practice clinics and of the least productive clinics to identify reasons for the differences. • Use low productivity scores as an indicator of potential for progress, and work with these clinics to help them improve their productivity. • Regress DEA scores on independent variables of interest, to determine causes of low productivity. • Use regression estimates to adjust clinics’ estimated productivity to account for causes beyond their control.

18. Thank you!