1 / 23

Supporting High-Stakes CMS Decision Making

Supporting High-Stakes CMS Decision Making. By Bruce Landon, Ph.D. Psychology Department Douglas College http://www.c2t2.ca/landonline. Memory Span Limits. The number of “things” that you can hold in your head at once while working on an problem

abedi
Download Presentation

Supporting High-Stakes CMS Decision Making

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Supporting High-Stakes CMS Decision Making By Bruce Landon, Ph.D. Psychology Department Douglas College http://www.c2t2.ca/landonline

  2. Memory Span Limits • The number of “things” that you can hold in your head at once while working on an problem • This limited “working memory” is a profound handicap for a rational decision maker • When you work with ideas in your head you are moving them around sort of like a juggler

  3. How Many?

  4. Even a Mathematician is limited by Memory Span

  5. Some are more limited

  6. Some can handle more

  7. But Seven – Nine is Maximum and Ten is only in your dreams

  8. When there are too many

  9. Some of them get dropped

  10. Or artistically set aside from the real mental consideration

  11. About Making Difficult Decisions: • Multi-Attribute Utility Theory, • Idealized Decision Process, • Cognitive Illusions, • Clues from Decision Making Research, • The Comparative Analysis Approach,

  12. Multi-Attribute Utility Theory: • Breaking a decision into independent dimensions • Determining the relative weights of each dimension • Listing of all of the alternatives • Ranking the alternatives along all dimensions (rating can work as well as ranking) • Multiplying the ranking by the weighting to determine the value • Selecting the alternative with the highest value

  13. Idealized Decision Process: • Select relevant features and assign importance weighting to features • Evaluate each application on relevant features and assign a suitability score • Score Applications by first multiplying each score by the corresponding feature weight • Select the application with the highest weighted average score - The Rational Choice

  14. Cognitive Illusions: • Availability Heuristic • Representativeness Heuristic • Hindsight Bias • Gambler's Fallacy • Effect of more options – delaying

  15. The Framing Effect • Refers to the frame of reference • People tend to avoid risks that are described in terms of benefits • But people tend to take risks described in terms of loss • Reminiscent of Win-stay, Lose-shift strategy

  16. Framing is like context for the size of the circle in the middle

  17. The Crowning fallibility is Overconfidence • The tendency to be more confident than is warranted by the evidence • To overestimate the accuracy of one's beliefs and judgments (availability heuristic again) • For example, the confidence of by the eye witness in their testimony is unrelated to the accuracy of that testimony • This overestimation of confidence enhances personal self-esteem and contributes to the resistance to being persuaded otherwise

  18. Comparative Analysis Approach: • Use review panel to provide consensus on feature/tool importance weighting • Limit Focus to what is required • Consider only a very few things at a time when making ratings/rankings of suitability • Make the computer keep track of the data and do the arithmetic calculations for the familiar weighted grading model for scores • Provide for sensitivity analysis (tweaking and recalculating)

  19. Small Example Decision model: • 1 Set weights, • 2 Evaluate parts, • 3 Select best score, • select best student • select best application • Link to www.c2t2.ca/landonline

  20. Making a Decision Policy with Decision Weights: • Simple strategies - ones and zeros • Complex hierarchical strategies - by user group then by function • Stakeholder involvement in setting importance weights • Peer Review Committee - with a distributed Delphi process • Opportunity to align the decision process with institutional values

  21. Review of Main Points: • Importance of the application selection decision • The cognitive illusions of the decision makers • Strategy to break down complex decision into: • smaller, simpler decisions • The Comparative Analysis Approach to Decisions • - Structure the decision with importance weights of important application features that accommodate your institutional context • - Rate suitability of single features/tools one at a time • - Use the Multi-Attribute Utility to select most suitable application for your institutional situation (highest weighted average among the candidates)

  22. Progress in redesigning landonline • Refocus on Higher Education products • New Advisory Board and WCET sponsor • Revised list of product features & glossary • Research Assistants for faster updates • Rewriting middleware as open source • Revised User Interface to Decisions • Companion sites to landonline.edutools.info • Rollout of sites in the summer 2002

  23. Thank you for Your Attention • Bruce_Landon@douglas.bc.ca

More Related