1 / 26

Sensitivity Analysis and Uncertainty

Sensitivity Analysis and Uncertainty. H. Scott Matthews 12-706/73-359/19-702 Lecture 6. Admin. HW 2 Due today Still no word on books We think they’re due today? Unfortunately cant keep pushing back lectures I will send e-mail with private link to PDF for Wednesday reading (can’t post).

melina
Download Presentation

Sensitivity Analysis and Uncertainty

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sensitivity Analysis and Uncertainty H. Scott Matthews 12-706/73-359/19-702 Lecture 6

  2. Admin • HW 2 Due today • Still no word on books • We think they’re due today? • Unfortunately cant keep pushing back lectures • I will send e-mail with private link to PDF for Wednesday reading (can’t post)

  3. Problem of Unknown Numbers • If we need a piece of data, we can: • Look it up in a reference source • Collect number through survey/investigation • Guess it ourselves • Get experts to help you guess it • Often only ‘ballpark’, ‘back of the envelope’ or ‘order of magnitude needed • Situations when actual number is unavailable or where rough estimates are good enough • E.g. 100s, 1000s, … (102, 103, etc.) • Source: Mosteller handout

  4. Definition: “Base Case” • Generally uses single values and our ‘best guesses’ • Sensitivity Analysis acknowledges uncertainty exists • Incorporate variables instead of constant assumptions • If our ‘Net Benefits’ remain positive over a wide range of reasonable assumptions, then robust results

  5. How many variables? • Choosing ‘variables’ instead of ‘constants’ for all parameters is likely to make model unsolvable • Partial sens. Analysis - change only 1 • Equivalent of dy/dx • Do for the most ‘critical’ assumptions • Can use this to find ‘break-evens’

  6. Best and Worst-Case Analysis • Analogous to “upper and lower bounds” used in estimation problems • Does any combination of inputs reverse the sign of our answer? • If so, are those inputs reasonable? • E.g. using very conservative ests. • Might want NB > 0, but know when NB < 0 • Similar to ‘breakeven analysis’

  7. Question 2.4 from Boardman • 3 projects being considered R, F, W • Recreational, forest preserve, wilderness • Which should be selected?

  8. Question 2.4 Project “R with Road” has highest NB

  9. Question 2.4 w/ uncertainty • What if we note that Benefits/Costs of each project are uncertain by plus or minus 10%? • e.g. instead of Project R having benefits of $10 million, could be as low as $9 million or as high as $11 million • Repeat for all project combinations • Now which project is ‘best’?

  10. Question 2.4 w/ uncertainty Best Case: R w/Road (same) Worst Case: Road Only General Case: Could be several But difficult to determine that from this chart - can we do better?

  11. Using error/uncertainty bars • Show ‘original’ point as well as range of uncertainty associated with point • Range could be fixed number, percentage, standard deviation, other • Excel tutorial available at: • http://phoenix.phys.clemson.edu/tutorials/excel/advgraph.html • See today’s spreadsheet on home page • Graphs original points, and min/max deviations from that as error bars…

  12. Error bar result Easier to see ‘best case/base case/worst case’ results - imagine moving a straight edge vertically up and down the axis to see result.

  13. Uncertainty • Investment planning and benefit/cost analysis is fraught with uncertainties • forecasts of future are highly uncertain • applications often made to preliminary designs • data is often unavailable • Statistics has confidence intervals – economists need them, too.

  14. Sensitivity Analysis (SA) • Most of our discussions and examples have been simple (e.g. y=mx+b) • Life is not as simple as this • E.g. y=ax+b+e • We need to be able to create models and methods that can incorporate our uncertainties • “SA” is just a fancy phrase for saying “tell me what happens to the decision if the inputs change”

  15. An Easy Visual Clue - Slider Bars • Use built-in excel functions to visually see effects of incremental changes in variables • “Scroll bars” (form toolbar) • Tutorial at: http://plato.acadiau.ca/aittmodules/excel/viewlets/scrollbar/scrollbar.viewlet/scrollbar_viewlet_swf.html • Let’s look at our old TV estimation problem

  16. EXCEL’s TABLE function • One- and two-input data tables • Sort of a built-in tool for sensitivity analysis (without fancy graphs/etc). • See PDF posted for examples and instructions

  17. Case: Photo-sensors for lighting • Save electricity by installing sensors in areas where natural light exists • Sensors ‘see’ light, only turn light fixtures on when needed • MAIN Posner 2nd floor hallway uses 106 15-watt fluorescent bulbs for 53 fixtures • How could we make a model to determine whether this makes sense? • Assume only one year time frame

  18. Photo-sensors for lighting • Assume we only care about ‘one year project’ • Costs = Labor cost, installation cost, electricity costs, etc. • Assume each bulb costs $6 • Benefits = ? How should we set up model? • Assume equal, set up as ‘show minimum cost’ option • Case 1 ‘Status quo’: assume lights used as is • On all the time, bulbs last 10,000 hours ~ burn out once per year) • Case 2 ‘PS’: pay to install sensors now, bulbs off between 1/3 and 1/2 of time

  19. Lighting Case Study - Status Quo • Costs(sq) - lights on all the time • Labor cost: cost of replacing used bulbs • “How many CMU facilities employees does it take to change a light bulb?” - and how long does it take? • Assume labor cost = $35/hr, 15 mins/bulb • 26.5 hours to change all bulbs each year, for a total labor cost of $927.50! Also, bulb cost $636/yr • Electricity: 106*15W ~ 14,000 kWh/yr (on 24-7) • Cost varies from 2.5 - 7.5 cents/kWh ~ $350-$1045 • Cost Replacing bulbs is same ‘order of magnitude’ as the electricity! (Total range [$1,911 - $2,608])

  20. Lighting Case Study - PS sensors • Costs(ps) - probably ‘off’ 1/3 - 1/2 of time • Labor cost: cost of installing sensors = ‘unknown’ • Labor cost: cost of installing new bulbs • Could assume 1/2 - 2/3 of bulbs changed per year instead of ‘all of them’ [Total $464 - $700] • Bulbs cost [$318-$424] • Electricity: 106*15W ~ 7,000 kWh/yr @ 1/2 • 9,333 kWh if off 1/3 of the time • Cost varies [ 2.5 - 7.5 cents/kWh] ~ $175-$700 • Total cost (w/o sensors) ~ [$955 - $1,740] • How much should we be WTP for sensors if time horizon is only one year?

  21. PS sensors analysis • WTP [$170 - $1,700] per year (NB>0) • We basically ‘solved for’ benefit • But our main sensitive value was elec. Cost, so range is probably [$919 - $1,129] per yr. • No overlap in ranges - PS always better • Should consider effects over several years • Could do a better bulb replacement model • Use more ranges - Bulb cost, labor, time • Check sensitivity of model answer to changes • Find partial sensitivity results for each • Look at spreadsheet model • This is fairly complicated - easier way?

  22. Sens. Analysis for Photo Sensors • Several built-in options from the plugins (@RISK-TopRank [Win only] or treeplan [Win/Mac]) to check sensitivity. • One-way (one variable at a time) • Two-way (two at a time) • Tornado (all at a time)

  23. One-Way Sensitivity Analysis • Use @RISK or treeplan plugin. • Makes graph that varies a single variable from a low- to high-end range and shows output (eg NPV) as a function of variable • The resulting graph shows how “sensitive” your answer is to changes in the one variable

  24. Tornado Diagrams • Shows results of many one-way plots on single chart • Length of bars tells you how sensitive output is to each variable • Bigger bar = more sensitive • Software typically “sorts” most to least • Looks like a tornado • Spider diagrams are similar

  25. Two-way SA • Treeplan plug-in cant do this • Shows a 2-dimensional plot of what happens when we change 2 variables at once • Graph generated is a “frontier”

  26. Plug-in notes • See Clemen pp.193- for TopRank tutorial • See course web page for instructions o using treeplan plugins (eg SensIt) • Read the PDFs for install instructions! • Should install ok on most cluster computers

More Related