DATA-DRIVEN APPOINTMENT SCHEDULING IN THE PRESENCE OF NO-SHOWS

1. DATA-DRIVEN APPOINTMENT SCHEDULING IN THE PRESENCE OF NO-SHOWS October 16 2012 Michele Samorani Linda LaGanga

2. The No-Show Problem • Mental Health Center of Denver (MHCD) • Large nonprofit organization • 36% of the appointments were no-shows! • MHCD can’t charge for not showing • MHCD already uses reminder calls • Progress in reducing no-shows for psychiatrist appointments • About 25% on average, and varies between doctors • What can they do?

3. Solution 1: Open Access • Show rate decreases if “lead time” increases • Give only same-day or next-day appointments • If too many patients call in for same-day appointments, defer them to tomorrow (Robinson and Chen 2010)

4. Solution 2: Overbooking p p p Regular Scheduling p • Compress slots (LaGanga and Lawrence 2007) 12:00 11:30 10:00 9:30 9:00 10:30 11:00 Overbooking p p p p p p p p p Lucky Case Low waiting time Low overtime 9:40 11:20 9:00 9:20 11:40 12:00 10:00 11:00 10:20 10:40 Overbooking p p p p p p p p p Unlucky Case High waiting time High overtime 9:40 11:20 9:00 9:20 11:40 12:00 10:00 11:00 10:20 10:40

5. Data-Driven Appointment Scheduling

6. Use Analytics to Schedule Appointments • Day-dependent show outcomes! • Lead time • Personal schedule • Day of week • Weather Show in day 0 Show in day 1 No-Show in day 2 Solve Scheduling Problem Classification Rule Day & Slot SSN Appointment Request Minimize overtime and waiting time Current Schedule Current day S S N Scheduling Horizon (h) S S

7. Goals • Understand the causes of no-shows (descriptive analytics) • Accurately predict show outcomes (predictive analytics) • Optimally schedule appointments (prescriptive analytics) • The scheduling policy must be practical (descr. + prescr. anlyt.) • Provide guidelines on clinic design (prescriptive analytics)

8. Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics) Classification Rule Solve Scheduling Problem Classification Rule Day & Slot Appointment Request

9. Classification Rule • Any Classification algorithm requires a mining table • Typically, the mining table is built manually • We build it automatically • Through Propositionalization (Samorani et al. 2011)

10. Propositionalization 1 Average Age of the clients seen in location 1 0..N 0..N 1 0..N Age of the client Average Age of the clients seen by the staff in location • Pick a path starting from Appointment • “Roll-up” attributes • Add a new attribute to the table Appointment • More than 3,000 attributes built in 3 hours!

11. New Knowledge! • What attributes are most discriminant? • Lead time • Location • Previous no-show rate • Service type • Staff characteristics: • Number of times they performed group therapy • Number of times they performed case management • Number of times at a certain location Expected Unexpected

12. Prediction Quality • Sensitivity= accuracy among the non-showing appointment requests • Specificity= accuracy among the showing appointment requests • Cost-sensitive classification to shift quality towards sensitivity or specificity Performance frontier LB LB LB LB LB LB LB LB LB LB LB UB Light grey: Random prediction Orange: Prediction quality at MHCD with Bayesian Network LB = lower bound; UB = upper bound

13. Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics) Solve the Scheduling Problem Solve Scheduling Problem Classification Rule Day & Slot Appointment Request

14. Mathematical Model • Patient categories for waiting time • Day- and patient-dependent revenues • Solved via column generation s.t.

15. Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics) Interpret the output of the scheduling algorithm Solve Scheduling Problem Classification Rule Day & Slot Appointment Request

16. Develop a Heuristic • Let’s analyze the output of the scheduling algorithm: • Target sequence: • A further analysis reveals that • No-shows tend to be scheduled far in advance • Shows tend to be scheduled in the near future -2.1% profit compared to optimal procedure • Heuristic: • Schedule predicted shows soon in S-slots • Schedule predicted no-shows far in the future in N-slots

17. Understand the causes of no-shows (descriptive analytics) Accurately predict show outcomes (predictive analytics) Optimally schedule appointments (prescriptive analytics) The scheduling algorithm must be interpretable (descr. + prescr. anlyt.) Provide guidelines on clinic design (prescriptive analytics) Guidelines on clinic design Solve Scheduling Problem Classification Rule Day & Slot Appointment Request

18. Sensitivity and Specificity regulate the trade-off between Patients seen and Wait. / Over times Sensitivity and Specificiy Same-day scheduling is worst if prediction quality is high *Bars and labels = profit

19. Comparison to Open Access • Open Access: same- or next-day scheduling without overbooking (Robinson and Chen 2010) • HP (sn, sp): heuristic procedure with sensitivity sn and specificity sp. Scheduling horizon = 5 days • 10,000 day simulations +18.0% +10.9% (benefit of analytics) • It can be shown that analytics is less beneficial for shorter scheduling horizons

20. At MHCD

21. Low show rate  Shift prediction quality to high sensitivity • High show rate  Shift prediction quality to high specificity

22. Benefits of Analytics Heuristic L. Bound *In selected MHCD clinics with low show rate, assuming capacity = 8, slots = 12

23. Benefits of Analytics Heuristic L. Bound *In selected MHCD clinics with low show rate, assuming capacity = 8, slots = 12

24. Conclusions

25. Contributions and Managerial Insights • Find causes of no-shows • Develop a dynamic scheduling algorithm that uses individual day-dependent no-show predictions • Develop an effective heuristic procedure that is interpretable and easy to implement • Find that same-day appointment is the worst policy if predictive analytics is used • Outperform open access by 18% at MHCD • Reduce system variability

26. Innovation in Analytics • Descriptive Analytics: • Propositionalization to find new knowledge • Predictive Analytics: • Cost-sensitive classification to favor one of the two conflicting objectives • Prescriptive Analytics: • Suggest when to lean towards sensitivity or specificity • Study the output of optimization through analytics

27. Implementation at MHCD • The implementation of the scheduling system at a MHCD clinic is currently in progress • First phase (DONE): implement an “observer” • Second phase: implement it in a real clinic

28. Thank you for your attention