1 / 21

Chapters 19 – 20: Dealing With Uncertainties

Chapters 19 – 20: Dealing With Uncertainties. TPS Example: Uncertain outcome Decision rules for complete uncertainty Utility functions Expected value of perfect information Statistical decision theory The value-of-information procedure. TPS Decision Problem 7. OS Development Options:

noel
Download Presentation

Chapters 19 – 20: Dealing With Uncertainties

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapters 19 – 20: Dealing With Uncertainties • TPS Example: Uncertain outcome • Decision rules for complete uncertainty • Utility functions • Expected value of perfect information • Statistical decision theory • The value-of-information procedure

  2. TPS Decision Problem 7 OS Development Options: • Option B-Conservative (BC) • Sure to work (cost=$600K) • Performance = 4000 tr/sec • Option B-Bold (BB) • If successful, Cost=$600K Perf.= 4667 tr/sec • If not, Cost=$100K Perf.= 4000 tr/sec Which option should we choose?

  3. Operating System Development Options

  4. Payoff Matrix for OS Options BB, BC

  5. Decision Rules for Complete Uncertainty • Maximin Rule. Determine minimum payoff for each option. Choose option maximizing the minimum payoff. • Maximax Rule. Determine max-payoff for each option. Choose option maximizing max-payoff. • Laplace Rule. Assume all states of nature equally likely. Choose option with maximum expected value. BB = 0.5(250) + 0.5(-50) = 100 BC = 0.5(250) + 0.5(50) = 50

  6. Difficulty with Laplace Rule U1 : performance failure U2 : reliability failure

  7. Breakeven Analysis • Treat uncertainty as a parameter P= Prob (Nature is favorable) • Compute expected values as f(P) EV (BB) = P(250) + (1-P)(-50) = -50 + 300P EV (BC) = P(50) + (1-P)(50) = 50 • Determine breakeven point P such that EV (BB) = EV (BC): -50 + 300P = 50, P=0.333 If we feel that actually P>0.333, Choose BB P<0.333, Choose BC

  8. Utility Functions Is a Guaranteed $50K The same as A 1/3 chance for $250K and A 2/3 chance for -$50K?

  9. Utility Function for Two Groups of Managers Utility 20 10 -3 -2 -1 1 2 3 4 5 Payoff ($M) -10 Group 1 -20 Group 2

  10. Possible TPS Utility Function 20 (250,20) (170,15) 10 (50,5) (100,10) 100 200 300 -100 -10 (-50,-20) -20 (-80,-25)

  11. Utility Functions in S/W Engineering • Managers prefer loss-aversion • EV-approach unrealistic with losses • Managers’ U.F.’S linear for positive payoffs • Can use EV-approach then • People’s U.F.’S aren’t • Identical • Easy to predict • Constant

  12. TPS Decision Problem 7 • Available decision rules inadequate • Need better information • Info. has economic value

  13. Expected Value of Perfect Information (EVPI) Build a Prototype for $10K • If prototype succeeds, choose BB Payoff: $250K – 10K = $240K • If prototype fails, choose BC Payoff: $50K – 10K = $40K If equally likely, Ev = 0.5 ($240K) + 0.5 ($40K) = $140K Could invest up to $50K and do better than before • thus, EVPI = $50K

  14. However, Prototype Will Give Imperfect Information That is, P(IB|SF) = 0.0 Investigation (Prototype) says choose bold state of nature: Bold will fail P(IB|SS) = 1.0 Investigation (prototype) says choose bold state of nature: bold will succeed

  15. Suppose we assess the prototype’s imperfections as P(IB|SF) = 0.20, P(IB|SS) = 0.90 And Suppose the states of nature are equally likely P(SF) = 0.50 P(SS) = 0.50 We would like to compute the expected value of using the prototype EV(IB,IC) = P(IB) (Payoff if use bold) +P(IC) (Payoff if use conservative) = P(IB) [ P(SS|IB) ($250K) + P(SE|IB) (-$50K) ] + P(IC) ($50K) But these aren’t the probabilities we know

  16. How to get the probabilities we need P(IB) = P(IB|SS) P(SS) + P(IB|SF) P(SF) P(IC) = 1 – P(IB) P(SS|IB) = P(IB|SS) P(SS) (Bayes’ formula) P(IB) P(SF|IB) = 1 – P (SS|IB) P(SS|IB) = Prob (we will choose Bold in a state of nature where it will succeed) Prob (we will choose Bold)

  17. 8 NET EV, $K 4 0 10 20 30 Net Expected Value of Prototype PROTO COST, $K

  18. Conditions for Successful Prototyping (or Other Info-Buying) • There exist alternatives whose payoffs vary greatly depending on some states of nature. • The critical states of nature have an appreciable probability of occurring. • The prototype has a high probability of accurately identifying the critical states of nature. • The required cost and schedule of the prototype does not overly curtail its net value. • There exist significant side benefits derived from building the prototype.

  19. Pitfalls Associated by Success Conditions • Always build a prototype or simulation • May not satisfy conditions 3,4 • Always build the software twice • May not satisfy conditions 1,2 • Build the software purely top-down • May not satisfy conditions 1,2 • Prove every piece of code correct • May not satisfy conditions 1,2,4 • Nominal-case testing is sufficient • May need off-nominal testing to satisfy conditions 1,2,3

  20. Statistical Decision Theory: Other S/W Engineering Applications How much should we invest in: • User information gathering • Make-or-buy information • Simulation • Testing • Program verification How much should our customers invest in: • MIS, query systems, traffic models, CAD, automatic test equipment, …

  21. The Software Engineering Field Exists Because Processed Information Has Value

More Related