Trade-offs in Explanatory Model Learning

1. 1 22nd of February 2012 Trade-offs in Explanatory Model Learning  DCAP Meeting Madalina Fiterau

2. 2 Outline • Motivation: need for interpretable models • Overview of data analysis tools • Model evaluation – accuracy vs complexity • Model evaluation – understandability • Example applications • Summary

3. Border control: vehicles are scanned Human in the loop interpreting results Example Application:Nuclear Threat Detection prediction feedback vehicle scan

4. 4 Boosted Decision Stumps • Accurate, but hard to interpret How is the prediction derived from the input?

5. 5 Decision Tree – More Interpretable yes no Radiation > x% no yes Payload type = ceramics Consider balance of Th232, Ra226 and Co60 yes no Uranium level > max. admissible for ceramics Threat Clear

6. 6 Motivation Many users are willing to trade accuracy to better understand the system-yielded results Need: simple, interpretable model Need: explanatory prediction process

7. 7 Analysis Tools – Black-box

8. 8 Analysis Tools – White-box

9. 9 Explanation-Oriented Partitioning 2 Gaussians Uniform cube (X,Y) plot

10. EOP Execution Example – 3D data Step 1: Select a projection - (X1,X2)

11. EOP Execution Example – 3D data Step 1: Select a projection - (X1,X2)

12. EOP Execution Example – 3D data h1 Step 2: Choose a good classifier - call it h1

13. EOP Execution Example – 3D data Step 2: Choose a good classifier - call it h1

14. EOP Execution Example – 3D data OK NOT OK Step 3: Estimate accuracy of h1 at each point

15. EOP Execution Example – 3D data Step 3: Estimate accuracy of h1 for each point

16. EOP Execution Example – 3D data Step 4: Identify high accuracy regions

17. EOP Execution Example – 3D data Step 4: Identify high accuracy regions

18. EOP Execution Example – 3D data Step 5:Training points - removed from consideration

19. EOP Execution Example – 3D data Step 5:Training points - removed from consideration

20. EOP Execution Example – 3D data Finished first iteration

21. EOP Execution Example – 3D data Finished second iteration

22. EOP Execution Example – 3D data Iterate until all data is accounted for or error cannot be decreased

23. Learned Model – Processing query [x1x2x3] [x1x2] in R1 ? yes h1(x1x2) no yes [x2x3] in R2 ? h2(x2x3) no yes [x1x3] in R3 ? h3(x1x3) no Default Value

24. Parametric / Nonparametric Regions Query point decision decision Correctly classified Incorrectly classified n3 n2 p n5 n1 n4

25. 25 EOP in context Similarities Differences Default classifiers in leafs CART Decision structure Models trained on all features Feating Local models Subspacing Low-d projection Keeps all data points Multiboosting Ensemble learner Committee decision Gradually deals with difficult data Boosting

26. 26 Outline • Motivation: need for interpretable models • Overview of data analysis tools • Model evaluation – accuracy vs complexity • Model evaluation – understandability • Example applications • Summary

27. 27 • Real valued features, binary output • Artificial data – 10 features • Low-d Gaussians/uniform cubes • UCI repository • Application-related datasets • Results by k-fold cross validation • Accuracy • Complexity = expected number of vector operations performed for a classification task • Understandability = w.r.t. acknowledged metrics Overview of datasets

28. EOP is often less accurate, but not significantly the reduction of complexity is statistically significant EOP vsAdaBoost - SVM base classifiers Accuracy Complexity Boosting EOP (nonparametric) mean diff in accuracy: 0.5% mean diff in complexity: 85 p-value of paired t-test: 0.832 p-value of paired t-test: 0.003

29. EOP (stumps as base classifiers) vs CART on data from the UCI repository EOP P. Accuracy Complexity • Parametric EOP yields the simplest models • CART is the most accurate CART EOP N.

30. Why are EOP models less complex? Typical XOR dataset

31. Why are EOP models less complex? • CART • is accurate • takes many iterations • does not uncover or leverage structure of data Typical XOR dataset

32. Why are EOP models less complex? • CART • is accurate • takes many iterations • does not uncover or leverage structure of data Typical XOR dataset • EOP • equally accurate • uncovers structure + o Iteration 1 o + Iteration 2

33. At low complexities, EOP is typically more accurate Error Variation With Model Complexity for EOP and CART EOP_Error-CART_Error Depth of decision tree/list

34. 35 UCI data – Accuracy White box models – including EOP – do not lose much

35. 36 UCI data – Model complexity White box models – especially EOP – are less complex Complexity of Random Forests is huge - thousands of nodes -

36. Accuracy-targeting EOP • Identifies which portions of the data can be confidently classified with a given rate. • Allowed to set aside the noisy part of the data. Robustness Accuracy of AT-EOP on 3D data Accuracy Max allowed expected error

37. 38 Outline • Motivation: need for interpretable models • Overview of data analysis tools • Model evaluation – accuracy vs complexity • Model evaluation – understandability • Example applications • Summary

38. Metrics of Explainability* * L.Geng and H. J. Hamilton - ‘Interestingness measures for data mining: A survey’

39. For 3 out of 4 metrics, EOP beats CART Evaluation with usefulness metrics BF =Bayes Factor. L = Lift. J = J-score. NMI = Normalized Mutual Info • Higher values are better

40. 41 Outline • Motivation: need for interpretable models • Overview of data analysis tools • Model evaluation – accuracy vs complexity • Model evaluation – understandability • Example applications • Summary

41. 42 Spam Detection (UCI ‘SPAMBASE’) • 10 features: frequencies of misc. words in e-mails • Output: spam or not Complexity

42. 43 • classifier labels everything as spam • high confidence regions do enclose mostly spam and: • Incidence of the word ‘your’ is low • Length of text in capital letters is high Spam Detection – Iteration 1

43. 44 Spam Detection – Iteration 2 • the required incidence of capitals is increased • the square region on the left also encloses examples that will be marked as `not spam'

44. 45 Spam Detection – Iteration 3 • Classifier marks everything as spam • Frequency of ‘our’ and ‘hi’ determine the regions word_frequency_hi

45. Example Applications Stem Cells ICU Medication identify combinations of drugs that correlate with readmissions explain relationships between treatment and cell evolution sparse features class imbalance Nuclear Threats Fuel Consumption identify causes of excess fuel consumption among driving behaviors support interpretation of radioactivity detected in cargo containers - high-d data - train/test from different distributions adapted regression problem

46. 47 Effects of Cell Treatment • Monitored population of cells • 7 features: cycle time, area, perimeter ... • Task: determine which cells were treated Complexity

47. 48

48. 49 Mimic Medication Data • Information about administered medication • Features: dosage for each drug • Task: predict patient return to ICU Complexity

49. 50

50. 51 Predicting Fuel Consumption • 10 features: vehicle and driving style characteristics • Output: fuel consumption level (high/low) Complexity