1 / 17

Mental Models of Cost Estimation: Assessing Uncertainty in Software Cost Modeling

This presentation discusses the use of mental models in cost estimation and uncertainty assessment in software cost modeling. It explores methodology for capturing and measuring changes in mental models, and implications for risk assessment.

zenia
Download Presentation

Mental Models of Cost Estimation: Assessing Uncertainty in Software Cost Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mental Models of Cost Estimation: A Focus on Uncertainty Assessment24th International Forum on COCOMO and Systems/Software Cost Modeling Dr. Ricardo Valerdi, MIT Dr. Linda Newnes, University of Bath Dr. Yee-May Goh, University of Bath Melanie Kreye, University of Bath Dr. Jairus Hihn, NASA JPL November 3, 2009

  2. Outline • Functions of mental models • Example mental model from NASA • Mental models of risk assessment • Methodology for capturing mental models • Measuring change in mental models • Implications & next steps

  3. © Northrop Grumman Mission Systems

  4. What is the value of π? • Physics: 3.14159265358979323946… • Math: c/d • COCOMO user: ~3.14 (round up to 4.0) • Accounting: 3.14 or whatever you want • Sales: 3.14 (before the mail-in rebate) • CEO: where’s my cake?

  5. Human Memory • Organized around schemas • Abstract and general rules based on experience and other people/information • When schemas are initiated, they direct behavior • In a novel situation, individuals recall certain schemas in order to make sense of the new information

  6. Mental Models Psychological representations of real, hypothetical or imaginary situations (Craik, K. The Nature of Explanation, 1943). • Mental models include what a person thinks is true, not necessarily what is actually true • Mental models are similar in structure to the thing or concept they represent • Mental models allow a person to predict the results of their actions • Mental models are simpler than the thing or concept they represent. They include only enough information to allow accurate predictions

  7. Functions of Mental Models Rouse, W. B., People and organizations: explorations of human-centered design , Wiley 2007.

  8. Example: Estimation Process at NASA(Normative Model) Source: Hihn, J., Griesel, A., Bruno, K., Fouser, T., Tausworthe, R., Mental Models of Software Forecasting, NASA JPL.

  9. Example: Estimation Process at NASA(Descriptive Model) Source: Hihn, J., Griesel, A., Bruno, K., Fouser, T., Tausworthe, R., Mental Models of Software Forecasting, NASA JPL.

  10. Mental Models of Risk Assessment Canadian Standards Association Source: Fischoff, B., Realistic Risk Disclosure in Newly Normal Times.

  11. Risk Assessment and Management Source: Cooper, L. P., How Project Teams Conceive of and Manage Pre-Quantitative Risks, PhD Dissertation, University of Southern California, 2008.

  12. Methodology for Capturing Mental Models • Semi-structured interviews (normative) • What models do you use to conceive of risk? • How is risk captured as part of the cost estimation activity? • Ethnography (descriptive) • Document descriptive risk perception based on spontaneous responses to certain problem situations • Differentiate risk perceptions across roles (cost estimator, PM, software engineer, etc.) and context (mission critical, IT development, telecom, etc.)

  13. Temporal Properties of Mental Models Immediate Past Future Source: Radioactive Waste Management Committee, Report No. NEA/RWM/FSC(2003)7/REV1, The Mental Models Approach to Risk Research.

  14. Measuring Change in Mental Models 1. Attain a high degree of experimental control 2. Separate measurement and improvement 3. Collect data from individuals in isolation 4. Collect detailed data from the memory of each individual 5. Measure change rather than perceived change 6. Obtain quantitative measures of characteristics of mental models 7. Employ a naturalistic task and response format 8. Obtain sufficient statistical power Doyle, J. K., Radzicki, M. J., Trees, W. S., Measuring the effect of systems thinking interventions on mental models, pp. 129-132 in George P. Richardson and John D. Sterman, eds. Proceedings of the 1996 International Conference of the System Dynamics Society. Cambridge, Massachusetts. July 22-25, 1996.

  15. Way Forward • Implications • Documenting mental models will help understand the schemas for pre-quantitative risk assessment • Richness of mental models will allow implicit information to influence quantitative risk assessment • Contextual differences in risk assessment can be made explicit • Next steps • Elicitation of mental models from a varied sample of engineering decision makers • Integration of uncertainty models with cost models

  16. Thank You

More Related