Thesis Proposal

1. Thesis Proposal Maximizing long-term ROI for Active Learning Systems

2. Interactive ClassificationGoal: Optimize life-time Return On Investment Majority transactions automatically cleared Learning Model to Flag Transactions for Manual Intervention Domain specific transaction processing Machine Learning model Large volume (in millions) of transactions coming in Transactions processed successfully Minority transactions flagged for auditing • Defining Characteristics • Expensive domain experts • Skewed class distribution (minority events) • Concept/Feature drift • Biased sampling of labeled historical data • Lots of unlabeled data Lower false positive rates based on learning model

3. Interactive Classification Applications • Fraud detection (Credit Card, Healthcare) • Network Intrusion detection • Video Surveillance • Information Filtering / Recommender Systems • Error prediction/Quality Control • Health Insurance Claims Rework

4. Health Insurance Claim Process - Rework Underpayments Overpayments

5. Why is solving Claims Rework important? • Inefficiencies in the healthcare process result in large monetary losses affecting corporations and consumers • For large (10 million+) insurance plan, estimated $1 billion in loss of revenue • $91 billion over-spent in US every year on Health Administration and Insurance (McKinsey study’ Nov 2008) • 131 percent increase in insurance premiums over past 10 years • Claim payment errors drive a significant portion of these inefficiencies • Increased administrative costs and service issues of health plans • Overpayment of Claims - direct loss • Underpayment of Claims – loss in interest payment for insurer, loss in revenue for provider

6. Interactive Classification Setting – Machine Learning Setup Trained Classifier Ranked List scored by classifier Unlabeled + Labeled Data • Classifier trained from labeled data • Human (user/expert) in the loop using the results but also providing feedback at a cost • Goal: Maximize long-term Return on Investment (equivalent to the productivity of the entire system)

7. Factorization of the problem Cost-Sensitive Exploitation Cost-Sensitive Active Learning Exploration-Exploitation Tradeoffs Standard Ranking / Relevance Feedback Active Learning

8. Factorization of the problem – characterization of the models • Uniform • Each instance has same value • Variable • Each instance has different value which is dependent on the properties of the instance • Markovian • Each instance has dynamically changing value depending on the (ordered) history of instances already observed, in addition to the factors for Variable model

9. Example Cases for Factorization of Cost Model • Uniform: Speculative versus definitive language usage distinction for biomedical abstracts • [Settles et al., 2008] • Variable: Part Of Speech tagging • Annotation time dependent on the sentence length with longer documents taking more time to label [Ringgeret al., 2008] • Markovian: Claims Rework Error Prediction • If similar claims are shown to the auditors in sequence reducing the cognitive switching costs, the time taken to label reduces [Ghani and Kumar, 2011]

10. Example Cases for Factorization of Exploitation Model • Uniform: Claims Rework Error Prediction • If we only account for the administrative overhead of fixing a claim [Kumar et al., 2010] • Variable: Claims Rework Error Prediction • If we take into account the savings based on the adjustment amount of the claim [Kumar et al., 2010] • Markovian: Claims Rework Error Prediction • Root cause detection [Kumar et al., 2010]

11. Example Cases for Factorization of Exploration Model • Uniform: Extracting contact details from email signature lines • Random strategy gives results comparable to other strategies [Settles et al., 2008] • Variable: KDD Cup 1999, Network Intrusion detection • Sparsity based strategy gives good performance [Ferdowsi et al., 2011] • Dependent on the properties of the examples (or population) which can be pre-determined. • Markovian: Uncertainty based active sampling strategy • Most commonly used strategy

12. Problem Statement How can we maximize long term ROI of active learning systems for interactive classification problems?

13. Proposed Hypothesis Jointly managing the cost, exploitation and exploration factors will lead to increased long term ROI compared to managing them independently

14. Proposed Contributions • A framework to jointly manage cost, exploitation and exploration • Extensions of Active Learning along the following dimensions • Differential utility of a labeled example • Dynamic cost of labeling an example • Tackling concept drift

15. Proposed Framework • Choice of Cost model • Choice of Exploitation model • Choice of Exploration model • Utility metric • Algorithms to optimize the utility metric

16. Choice of Models Markovian Exploitation Model Variable Uniform Uniform Variable Markovian Variable Markovian Cost Model Exploration Model Uniform

17. Utility Metric • Domain dependent • May or may not have a simple instantiation in the domain • Possible instantiations for Claims Rework domain • Return on Investment (Haertal et al, 2008) • Corresponds to the business goal of the deployed systems • Return: Cumulative dollar value of claims adjusted • Investment: Cumulative time (equivalent dollar amount) for auditing the claims • Does not take into account the classifier improvement/degradation • Amortized Return on Investment • Amortized return: Calculate the net present value of the returns based on the expected future classifier improvement • Return: Cumulative dollar value of claims adjusted + net present value of the increased returns due to future classifier improvement • Investment: Cumulative time (equivalent dollar amount) for auditing the claims • Takes into account exploration and exploitation

18. Algorithm to optimize the utility metric • Optimization straightforward if a well defined utility metric exists for the domain • Computational approximations may still be required for practical feasibility • Cases where a utility metric is not well defined based on the constituent cost/exploration/exploitation models, approaches to explore • Rank fusion based approach • Each model provides a ranking which are combined to get a final ranking • Explore relevant approaches from reinforcement learning • Exploration-exploitation tradeoff using variance estimates in multi-armed bandits (Szepesvári et al, 2009) • Upper Confidence Bounds for Trees (Kocsis and Szepesvári, 2006) • Multi-armed bandit with dependent arms (Pandey et al, 2007)

19. Interactive Classification Framework-Experimental Setup Unlabeled Data (t) Trained Classifier (1,…,t-1) Labeled Data (1,…,t-1) Ranked List Labeled Data (t) Performance evaluation done on the set of labeled instances obtained at each iteration

20. Evaluation Compare various approaches with multiple baselines • Random • Pure Exploitation • Exploitation=Var; Exploration=Uniform; Cost=Uniform • Pure Exploration • Exploration=Var/Mar; Exploitation=Uniform; Cost=Uniform • Pure Cost sensitive • Cost=Var; Exploitation=Uniform; Exploration=Uniform

21. Preliminary results • Graph with results from framework

22. Generalizing Active Learning for Handling Temporal Drift • What is temporal drift? • Changing data distribution • Changing nature of classification problem • Adversarial actions • Related Work • Traditional active learning assumes static unlabeled pool • Stream-based active learning (Chu et al., 2011) assumes no memory to store the instances and makes online decisions to request labels • Not completely realistic as labeling requires human effort and is usually not real-time • Learning approaches from data streams with concept drift predominantly use ensembles over different time period (Kolter and Maloof, 2007)

23. Proposed Setup for Temporal Active Learning • Periodically changing unlabeled pool, corresponding to the experimental setup for interactive framework • Cumulative streaming pool • Recent streaming pool • Novel setup • Three components for handling temporal drift • Instance selection strategy • Type of model: Ensemble or Single • Instance or model weighing scheme

24. Proposed Instance Selection Strategies • Model Weight Drift Strategy • Feature Weight Drift Strategy • Feature Distribution Drift Strategy

25. Detecting Drift – Change in Models over Time • Claims rework domain • 15 models built over 15 time periods • Similarity between the models based on cosine measure

26. Preliminary results • Evaluation metric: Precision at 5 percentile • Represented in graph as percentage of the best strategy at a given iteration to give a sense that the mentioned strategies are not the best strategies at all iterations • Uncertainty begins to perform poorly at later iterations and feature drift based strategy starts performing better

27. Proposed Work • More experiments and analysis for claims rework data with data from different clients • More experiments based on synthetic dataset with longer observation sequence to analyze the performance of sampling strategies • Generation of synthetic data based on Gaussian Mixture models to mimic real data

28. Cost-Sensitive Exploitation Cost-Sensitive Exploitation

29. More Like This strategy Select Top m% claims Cluster Labeled Data Rank Ranked List scored by classifier Online Strategy

30. Online “More-like-this” Algorithm Require a labeled set L and an unlabeled set U • Train classifier C on L • Label U using C • Select top m% scored unlabeled examples UT • Cluster the examples UTU L into k clusters • Rank the k clusters using a exploitation metric • For each cluster ki in k • Rank examples in ki • For each example in ki • Query expert for label of • If precision of cluster kiis < Pmin and number of labels > Nmin, Next

31. Offline Comparison – MLT vs Baseline • 9% relative improvement over baseline for Precision at 2nd percentile metric

32. Live System Deployment ~$10 Million savings /year for a typical insurance company • Number of claims audited: • Baseline system: 200 • More-Like-This: 307 • 90% relative improvement over baseline • 27% reduction in audit time over baseline

33. SummaryProblem Statement How to maximize long term ROI of active learning systems for interactive classification problems

34. SummaryThesis Contributions • Characterization of the interactive classification problem • Defining the cost/exploration/exploitation models • Uniform • Variable • Markovian • Generalization (Extensions?) of Active Learning along the following dimensions • Differential utility of a labeled example • Dynamic cost of labeling an example • Tackling concept drift • A framework to jointly manage these considerations

35. SummaryEvaluation • Empirical Evaluation of the proposed framework • Using evaluation metric motivated by real business tasks • Datasets • Real world dataset: Health Insurance Claims Rework • Synthetic dataset • Comparison with multiple baselines based on underlying cost/exploitation/exploration models • Methodological contribution • Novel experimental setup • Intend to make the synthetic dataset and its generators public

36. SummaryProposed Work: Temporal Active Learning • Creation of synthetic datasets • Evaluation and analysis of proposed strategies on synthetic and claims rework dataset

37. SummaryProposed Work: Framework for interactive classification • Evaluate multiple utility metrics/optimization algorithm for Claims Rework domain • Augment temporal drift synthetic data for evaluating framework • Evaluate multiple utility metrics/optimization algorithm for synthetic dataset Markovian Exploitation Model Variable Uniform Uniform Variable Markovian Variable Markovian Uniform Cost Model Exploration Model

38. Thanks

39. Problem Description • High level factorization of the problem • Related Work • Triangle • Our proposed approach – framework • Broad categorization of the models • Choice of models • Choice of utility metric • Choice of optimization • Proposed work (various aproaches) • Temporal active learning • Some initial results • Cost sensitive exploitation • Summary • Problem statemnt • Contributions • Evaluation

40. Thesis Contributions • Problem Statement: How to generalize active learning to incorporate differential utility of a labeled example(dynamic/variable exploitation), dynamic cost of labeling an example, concept drift in a unified framework that makes the deployment of such learning systems practical • Contributions • Characterization of the interactive learning problem • Generalization of Active Learning along the following dimensions • Differential utility of a labeled example • Dynamic cost of labeling an example • Tackling concept drift • Cost-Sensitive Exploitation • A unified framework to solve these considerations jointly • First solution: Optimizing joint utility function based on cost, exploration utility and exploitation utility • Second solution: Using Upper Confidence Bound approach with contextual multi-armed bandit setup to incorporate the different factors • Empirical Evaluation of the proposed framework • Using evaluation metric motivated by real business tasks • Datasets • Synthetic dataset • Real world dataset: Health Insurance Claims Rework • Comparison with multiple baselines based on underlying factors

41. Situating the thesis work wrt related work • Efficiency & Representation • Feature level feedback • Feature acquisition • Batch active learning Cost-sensitive Active Learning • PrActive • Learning • Differential Utility • Dynamic cost • Concept Drift • Proactive • Learning • Unreliable Oracle • Oracle variation

42. Problem Statement How to generalize active learning to incorporate differential utility of a labeled example(dynamic/variable exploitation), dynamic cost of labeling an example, concept drift in a unified framework that makes the deployment of such learning systems practical