1 / 33

Towards COSYSMO 2.0 Future Directions and Priorities CSSE Annual Research Review Los Angeles, CA

Towards COSYSMO 2.0 Future Directions and Priorities CSSE Annual Research Review Los Angeles, CA. March 17, 2008 Garry Roedler Gan Wang Jared Fortune Ricardo Valerdi. Agenda. Context setting Discussion on COSYSMO 2.0 improvements Prioritization exercise. Context setting.

cole
Download Presentation

Towards COSYSMO 2.0 Future Directions and Priorities CSSE Annual Research Review Los Angeles, CA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards COSYSMO 2.0Future Directions and PrioritiesCSSE Annual Research ReviewLos Angeles, CA March 17, 2008 Garry Roedler Gan Wang Jared Fortune Ricardo Valerdi

  2. Agenda • Context setting • Discussion on COSYSMO 2.0 improvements • Prioritization exercise

  3. Context setting

  4. How is Systems Engineering Defined? • Product Realization • Implementation Process • Transition to Use Process • Technical Evaluation • Systems Analysis Process • Requirements Validation Process • System Verification Process • End Products Validation Process • Acquisition and Supply • Supply Process • Acquisition Process • Technical Management • Planning Process • Assessment Process • Control Process • System Design • Requirements Definition Process • Solution Definition Process EIA/ANSI 632, Processes for Engineering a System, 1999. Note: The requirements of EIA/ANSI 632 are addressed in ISO/IEC 15288, which was also used as a Source for consistent definition in COSYSMO.

  5. COSYSMO Origins Current SE Standards EIA-632 ISO/IEC 15288 Systems Engineering (SE) (Warfield 1956, EIA 1999, ISO/IEC 2002) 2000 1950 Software Cost Modeling COSYSMO (Boehm 1981) 1980 SW-CMM® SE-CMM ® SECM CMMI® (Humphrey 1989) 1990 2000 *CMM and CMMI are registered trademarks of Carnegie Mellon University Warfield, J. N., Systems Engineering, United States Department of Commerce PB111801, 1956. Boehm, B. W., Software Engineering Economics, Prentice Hall, 1981. Humphrey, W. Managing the Software Process. Addison-Wesley, 1989. EIA/ANSI 632, Processes for Engineering a System, 1999 ISO/IEC 15288, System Life Cycle Processes, 2002.

  6. COSYSMO Data Sources

  7. Modeling Methodology 3 rounds; > 60 experts 62 data points; 8 organizations

  8. COSYSMO Scope • Addresses first four phases of the system lifecycle (adapted from ISO/IEC 15288) • Considers standard Systems Engineering Work Breakdown Structure tasks (per EIA/ANSI 632) Conceptualize Operate, Maintain, or Enhance Replace or Dismantle Transition to Operation Oper Test & Eval Develop

  9. COSYSMO Operational Concept # Requirements # Interfaces # Scenarios # Algorithms + 3 Adj. Factors Size Drivers COSYSMO Effort Effort Multipliers • Application factors • 8 factors • Team factors • 6 factors Calibration

  10. COSYSMO Model Form Where: PMNS = effort in Person Months (Nominal Schedule) A = calibration constant derived from historical project data k = {REQ, IF, ALG, SCN} wx = weight for “easy”, “nominal”, or “difficult” size driver = quantity of “k” size driver E = represents diseconomies of scale EM = effort multiplier for the jth cost driver. The geometric product results in an overall effort adjustment factor to the nominal effort.

  11. Size Drivers vs. Effort Multipliers • Size Drivers: Additive, Incremental • Impact of adding a new item inversely proportional to current size • 10 -> 11 rqts = 10% increase • 100 -> 101 rqts = 1% increase • Effort Multipliers: Multiplicative, system-wide • Impact of adding a new item independent of current size • 10 rqts + high security = 40% increase • 100 rqts + high security = 40% increase

  12. Size Driver Weights

  13. UNDERSTANDING FACTORS Requirements understanding Architecture understanding Stakeholder team cohesion Personnel experience/continuity COMPLEXITY FACTORS Level of service requirements Technology Risk # of Recursive Levels in the Design Documentation Match to Life Cycle Needs OPERATIONS FACTORS # and Diversity of Installations/Platforms Migration complexity Cost Driver Clusters • PEOPLE FACTORS • Personnel/team capability • Process capability • ENVIRONMENT FACTORS • Multisite coordination • Tool support Criteria + Matched driver polarity + Grouped by theme + Combined moderately correlated parameters

  14. Cost Driver Rating Scales EMR = Effort Multiplier Ratio

  15. Cost Drivers Ordered by Effort Multiplier Ratio (EMR)

  16. EIA/ANSI 632 Effort Profiling Operate, Maintain, or Enhance Transition to Operation Operational Test & Evaluation Replace or Dismantle Conceptualize Develop Life Cycle Phases/Stages Acquisition & Supply Technical Management System Design Product Realization Technical Evaluation

  17. Impact 10 theses Academic Curricula Model Academic prototype Commercial Implementations Intelligence Community Sheppard Mullin, LLC Policy & Contracts Proprietary Implementations SEEMaP COSYSMO-R SECOST

  18. COSYSMO 2.0 Improvements

  19. Recommended Improvements(from user community) • Reuse • Integration of SwE & SysE estimation • Assumption of linearity in COSYSMO cost drivers • Effect of cost drivers and scale factors • Number of recursive levels of design • Risk modeling • Establishing best practice guidance • Consideration of SoS scope in COSYSMO • Estimation in Operation & Maintenance Phase • Requirements volatility Joint meeting Deferred

  20. 1. Reuse • Central question: What is the effect of reuse in estimating systems engineering size/effort? • Hypothesis: A COSYSMO reuse submodel will improve the model’s estimation accuracy • POC: Jared Fortune • References • Valerdi, R., Wang, G., Roedler, G., Rieff, J., Fortune, J., “COSYSMO Reuse Extension,” 22nd International Forum on COCOMO and Systems/Software Cost Modeling, 2007.

  21. 2. Integration of SwE & SysE estimation • Central question: What is the overlap between COCOMO II and COSYSMO? • Hypothesis: By identifying the WBS elements in COSYSMO that overlap with the WBS in COCOMO II, the systems engineering resource estimation accuracy increases • POC: Ricardo Valerdi • References • Valerdi, R., The Architect and the Builder: Overlaps Between Software and Systems Engineering. (working paper)

  22. 3. Linearity in COSYSMO cost drivers • Central question: How do we characterize the non-linearity of cost drivers across the system life cycle? • Hypothesis: Not all cost drivers have a constant impact on systems engineering effort throughout the life cycle. • POC: Gan Wang • References • Wang, G., Valerdi, R., Boehm, B., Shernoff, A., “Proposed Modification to COSYSMO Estimating Relationship,” 18th INCOSE Symposium, June 2008.

  23. 4. Effect of cost drivers and scale factors • Central question: Can some of the cost drivers become scale factors in the cost estimating relationship calibrated by the new data set? • Hypothesis: The current set of size and cost drivers are too sensitive to small variations in rating levels. • POC: Gan Wang • References • Wang, G., Valerdi, R., Boehm, B., Shernoff, A., “Proposed Modification to COSYSMO Estimating Relationship,” 18th INCOSE Symposium, June 2008. Note from GJR: Potential relationship between this and improvement #7. Also, in reality, number of recursive levels has a different impact on the effort than other cost drivers. This may be able to be addressed by this improvement, by improvement #5, or could be a separate improvement depnding on scope of each. Should be an item for discussion as we go through these.

  24. 5. Number of recursive levels of design • Central question: How can the integration complexity of system elements one layer below the system-of-interest be operationalized? • Hypothesis: The integration complexity of system elements is a predictor of systems engineering effort. • POC: John Rieff • References • Marksteiner, B., “Recursive Levels and COSYSMO”, October 2007. (working paper)

  25. 6. Risk Modeling • Central question: How can risk associated with the COSYSMO estimate be quantified? • Hypothesis: The output generated by COSYSMO can be quantified using probability distributions for better assessment of the likelihood of meeting the estimate • POC: John Gaffney (developer of COSYSMO-R) • References • Valerdi, R., Gaffney, J., “Reducing Risk and Uncertainty in COSYSMO Size and Cost Drivers: Some Techniques for Enhancing Accuracy,” 5th Conference on Systems Engineering Research, March 2007, Hoboken, NJ.

  26. 7. Best practice guidance for use of Cost Drivers • Central question: How can misuse of the COSYSMO cost drivers be avoided? • Hypothesis: By developing a best practice guide that describes common pitfalls associated with COSYSMO cost drivers, over-estimation can be reduced or avoided • POC: Garry Roedler • References • COSYSMO User Manual

  27. 8. Consideration of SoS scope in COSYSMO • Central question: How can COSYSMO be updated to address system of systems effort estimation? • Hypothesis: To be discussed in joint session • POC: Jo Ann Lane

  28. 9. Estimation in Operation & Maintenance Phase • Central question: How can we estimate systems engineering effort in the Operate & Maintain phase? • Hypothesis: Coverage of the Operate & Maintenance phases will broaden to model’s life cycle coverage • POC: Ricardo Valerdi

  29. 10. Requirements volatility • Central question: How do we quantify the effects of requirements volatility on systems engineering effort throughout the life cycle? • Hypothesis: Requirements volatility is a significant factor for predicting systems engineering effort and can serve as a leading indicator for project success • POC: Ricardo Valerdi • Feb 15, 2007 Workshop led by Rick Selby • Identified critical success factors in: technical, product, process, people • http://sunset.usc.edu/events/2007/ARR/presentations/RequirementsVolatilityWorkshopSummaryARR2007.ppt • Loconsole, A., Borstler, J., “An industrial case study on requirements volatility measures,” 12th Asia-Pacific Software Engineering Conference, 2005.

  30. Prioritization Exercise • Factors to Consider • Availability of data • Impact on total cost of ownership • Frequency of use • Compatibility with other models (i.e., COCOMO family, PRICE-H, etc.) • Addressal of future trends (Volatility, Uncertainty, Scalability) • Factor interactions

  31. Recommended Improvements(from user community) • Reuse (completed and approved for V2.0 baseline – see minutes from workshop at PSM User Conference) • Integration of SwE & SysE estimation • Assumption of linearity in COSYSMO cost drivers • Effect of cost drivers and scale factors • Number of recursive levels of design • Risk modeling (completed and approved for V2.0 baseline – see minutes from workshop at PSM User Conference) • Establishing best practice guidance • Consideration of SoS scope in COSYSMO • Estimation in Operation & Maintenance Phase • Requirements volatility Joint meeting Deferred

  32. Priority

More Related