1.18k likes | 1.37k Views
Decisions Under Incomplete Representations & Reasoning Some Reflections Eric Horvitz Microsoft Research SADR Meeting on Decision Making in Complex Environments Cornell University April 5, 2003 horvitz@microsoft.com http://research.microsoft.com/~horvitz/. Overview.
E N D
Decisions Under Incomplete Representations & ReasoningSome ReflectionsEric HorvitzMicrosoft ResearchSADR Meeting on Decision Making in Complex EnvironmentsCornell UniversityApril 5, 2003horvitz@microsoft.comhttp://research.microsoft.com/~horvitz/
Overview • Decision-theoretic systems in the real world • Decision support • Autonomous decision making, embedded processes • Some successes • Decision making under bounded resources • Inference and modeling • Systems immersed in environments Eric Horvitz, April 5, 2003
Fuel Battery p(F) p(B) Gauge p(G|F,B) TurnOver Start p(T|B) p(S|F,T) Graphical Models and Decision Systems • Twenty years of accelerated results, innovation • Bayesian networks and influence diagrams • Parsimonious representation of joint distribution • Theory of independence, d-separation Eric Horvitz, April 5, 2003
D A A R R A D A R D D R D D R Handcrafted and Learned Expert Model Data Learned Model Eric Horvitz, April 5, 2003
Representing Problems of Action • Influence diagrams (Howard & Matheson, 1975) Action A Utility World State H E3 E2 E1 En Eric Horvitz, April 5, 2003
FUTURE CORONARY ARTERY ARTERY STATUS STATUS FUTURE CHEST PAIN CHEST PAIN MYOCARDIAL INFARCTION LIFE QUALITY HEART SURGERY REMAINING LIFE VALUE YEARS MONETARY COST Representing Multiple Dimensions of Value Eric Horvitz, April 5, 2003
Plan {(A1,t1)...(An,tn)} Utility An, tn A3, t3 A1, t1 State of System H, t1 State of System H, tn State of System H, t2 En, t1 En, t4 E2, t1 E2, t4 En, t2 E1, t1 E2, t2 E1, t4 E1, t2 Eric Horvitz, April 5, 2003 Beyond Single Shot Decisions • Sequences of inference, actions, sensing under uncertainty
Reward (t+1) Reward t Action(t+1) Action(t) Observe t+1 Observe t State t+1 State t State t+2 Eric Horvitz, April 5, 2003 Partially Observable Markov Decision Process Representation
Real-World Applications Eric Horvitz, April 5, 2003
Scaling Up Bayesian Networks PULMONARY EMBOLUS HYPOVOLEMIA LV FAILURE ANAPHYLAXIS ANESTHESIA INSUFFICIENT KINKED TUBE SHUNT PAP DISCONNECTION INTUBATION STROKE LVED HISTORY VOLUME VOLUME LV FAILURE VENT MACHINE CATECHOLAMINE VENT ALV CARDIAC VENT SAO2 TPR OUTPUT TUBE VENT LUNG CVP PCWP HEART PA SAT RATE MV SETTING BLOOD PRESSURE ERROR MINUTE CAUTER ERROR VENTILATION FIO2 PRESSURE LOW OUTPUT ARTERIAL CO2 EXPIRED HR BP HR EKG HR SAT CO2 Eric Horvitz, April 5, 2003
Chief Complaint: Abdominal Pain Eric Horvitz, April 5, 2003
Pulomonary/Sleep ApneaIntegration with Physiological Monitoring Eric Horvitz, April 5, 2003
Fault Diagnosis and Troubleshooting Eric Horvitz, April 5, 2003
On the Web… www.microsoft.com Eric Horvitz, April 5, 2003
Very long Very short Control of Computation: Restarts in Satistifiabilty Solvers Great variation in execution time • Very short and very long runs for different randomized runs on same instances • Heavy-tailed distribution (Pareto) Eric Horvitz, April 5, 2003
100-1000x Decisom Model for Dynamic Restart Policies • To date: success with simple fixed policy: Restart search if run-time is greater than x • Orders of magnitude speedup Time to Solution Time expended before restart Gomes, et al. 1997 Eric Horvitz, April 5, 2003
Learning Bayesian Models + Decision-Theoretic Analysis • Features • Dynamic and static Gomes, et al. 1997 Eric Horvitz, April 5, 2003
Eric Horvitz, April 5, 2003 Horvitz, et al. 2001
Human-Computer Interaction User’s Profile User’s Goals User’s Needs User Activity Eric Horvitz, April 5, 2003
Monitoring Space Shuttle Telemetry Horvitz & Barry, 95 Eric Horvitz, April 5, 2003
Assistance with Software User background Primary goal Hierarchical presentation Consolidation Pivot wizard Chart wizard Repeated chart create/delete Group mode Database defined 3D cell reference Leading spaces External reference Repeated chart change Use query Repeated print / hide Rows Multicell selection Adjacent conceptual granularity Eric Horvitz, April 5, 2003 Horvitz, et al., 98
Profile Profile Inference about a User’s Time-Dependent Goals Profile Goalt-n Goalto Goalt-1 Ei,t-n Ej,t-n Ei,to Ei,t-1 Ej,to Ej,t-1 Time Horvitz, et al. 98 Eric Horvitz, April 5, 2003
Information Filtering & Alerting Horvitz, et al., 03 Eric Horvitz, April 5, 2003
Reasoning about Attention Horvitz, et al., 03 Eric Horvitz, April 5, 2003
Horvitz, et al., 03 Eric Horvitz, April 5, 2003
Decision-Theoretic Control of Dialog Horvitz & Paek, 01 Eric Horvitz, April 5, 2003
Harnessing Multiple Classes of Evidence • Speech recognizer confidence • Correctness of natural language parse as evidence • Visual analysis of pose Paek & Horvitz, 00 Eric Horvitz, April 5, 2003
ASR Reliability Indicator(t-1) Intention, Attention, and Clarification Dialog Dialog or Domain-Level Action(t-1) Dialog or Domain-Level Action(t) Utility(t-1) Utility(t) Context Context Speaker’s Goal(t-1) Speaker’s Goal(t) User’s Spoken Intention(t-1) User’s Spoken Intention(t) Content at Focus (t-1) Content at Focus (t) User Actions(t-1) User Actions(t) ASR Reliability Indicator(t-1) ASR Candidate n Confidence(t-1) ASR Candidate 1 Confidence(t-1) ASR Candidate n Confidence(t) ASR Candidate 1 Confidence(t) . . . . . . Horvitz & Paek, 01 Eric Horvitz, April 5, 2003
yes Engage Repeat reflect Tshoot Expected utility of actions Inferred beliefs Horvitz & Paek, 01 Eric Horvitz, April 5, 2003
overheard Tshoot yes Repeat no Engage reflect Interaction Troubleshooting Horvitz & Paek, 01 Eric Horvitz, April 5, 2003
Assessing Preferences on Outcomes Horvitz & Paek, 01 Eric Horvitz, April 5, 2003
Assessing Frustration with Number of Steps Horvitz & Paek, 01 Eric Horvitz, April 5, 2003
But Still Cannot Solve Important Problems • Intractability per problem size and structure • Little experience with fielding autonomous, immersed decision making systems • Poor understanding of subobtimality associated with incomplete models and inference Toward principles of rational action under resource limitations Eric Horvitz, April 5, 2003
Hardness • Inference is NP-Hard; approximation is NP-Hard • Work on a tapestry of exact and approximate algorithms • Exploit special structure, probe portions of model, restricted probabilistic relationships Eric Horvitz, April 5, 2003
Quickscore TopN S Heckerman 1989 Henrion 1988, 1990 Shwe & Cooper 1990 For QMR-DT network A Tapestry of Approximation Algorithms Approximate methods Exact methods Polytrees Stochastic Search Kim & Pearl, 1983 simulation methods Arc reversal and Branch and bound reduction Olmsted, 1983 Logic sampling Shachter, 1986, 1988. NESTOR Markov simulation Forward propagation Loop Cutset Cooper, 1984 Henrion, 1986 Gibbs sampling conditioning : Set-covering: Pearl, 1987 Pearl, 1986 Peng & Reggia, Moralizing and propag- 1987 ation in a tree of cliques Lauritzen & Spiegelhalter, 1988 FPRAS: Likelihood Importance Bounded conditioning Chavez & Clustering weighting sampling Horvitz and Suermondt, Cooper, Pearl 1988 1988 1989 PIBNET is NP-hard Cooper, 1987 Fung & Shachter & Selective Chang 1989 Peot 1989 conditioning Dagum and Horvitz, 1992 Eric Horvitz, April 5, 2003
Reasoning Across Internal Medicine • QMR-DT: Internal Medicine QMR-DT derived from Internist-1/ QMR KB 534 diseases 40740 arcs 4040 findings Eric Horvitz, April 5, 2003
Toward Principles • What do we do when we cannot solve a decision problem completely? • How do we trade computation time and value of computation? • What is a complete problem? • How do we reason about sensitivity of the MEU of action to refinement effort? • Toward principles of decisions under bounded resources • Simon: Inadequacy of MEU • Good: Type II Rationality Eric Horvitz, April 5, 2003
Implementing Type II RationalityUnder Limited Resources • Need tractable meta-analysis of ideal nature and extent of reasoning • Need characterizable approximation methods • Need good understanding of cost of time, memory Eric Horvitz, April 5, 2003
Understanding Cost of Computation • “Cost:” “… … - C(t)” • Influence of specific classes of cost on value of computation, comprehensive value of ultimate action • Active inference about the structure of cost, expectation Eric Horvitz, April 5, 2003
Cost of Computational Resources Multiple cost contexts Urgency Deadline Urgent Deadline Uncertain Deadline Target • Urgency • Target • Deadline • Uncertain deadline Eric Horvitz, April 5, 2003 Horvitz, 88
Time Criticality for Shuttle Propulsion Barry and Horvitz (1993) 1.0 p(Engine Damaged) 0.5 p(Shrapnel) 0 1 2 3 4 5 Seconds Continue During Propellant Failure * p(Vehicle Lost | Engine Damaged, No Shrapnel,C1) = .5 * p(Vehicle Lost | Engine Damaged, Shrapnel, C1) = .8 Horvitz & Barry, 95 Eric Horvitz, April 5, 2003
1.0 • Minor Fracture • Hollow viscus Pr(Survive) • Solid organ injury • Major fracture • Intracranial hemorrhage • Major respiratory • Major vascular • Major airway 0.0 0 30 90 60 Delay of emergency center care (minutes) Inferring Costs of Delay • Case of transportation planning for trauma care Horvitz & Seiver, 97 Eric Horvitz, April 5, 2003
Context-Sensitive Costs Eric Horvitz, April 5, 2003 Horvitz & Seiver, 97
Characterizing the Value of Computation Eric Horvitz, April 5, 2003
Idealization.. Utility • Immediate value of result • Net value of result t • Cost of delay Horvitz, 90 Eric Horvitz, April 5, 2003
Object value Two opt, nearest neighbor • • Utility Net value 30 cities 5 10 15 20 25 30 35 40 45 50 55 60 Time (seconds) Horvitz & Klein, 91 Cost of delay Horvitz & Klein, 91 Eric Horvitz, April 5, 2003
Available idle time Uncertain Quality, Cost, Resources Utility • Immediate value of result • Net value of result t Horvitz, 90 Eric Horvitz, April 5, 2003
Models of Type II RationalityExample: Acute Respiratory Failure (ARF) vs. Respiratory Distress (RD) Horvitz, 90 Eric Horvitz, April 5, 2003
Example:Acute Respiratory Failure (ARF) vs. Respiratory Distress (RD) Actions ARF No ARF A: Treat ARF Treat ARF u(A,S) u(A,S) A: Treat RD States S: ARF Treat RD u(A,S) u(A,S) S: RD Horvitz, 90 Eric Horvitz, April 5, 2003
1.0 0.8 0.6 Probability 0.4 0.2 0.0 0 10 20 30 40 Time (Subproblems solved) Bounding Conditioning Approximation:Acute Respiratory Failure vs. Respiratory Failure p(S|E1… En) Eric Horvitz, April 5, 2003 Horvitz, et al. 89