COSATMO, COCOMO III, and COSYSMO: Developing Next-Generation Cost Models

1. COSATMO, COCOMO III, and COSYSMO:Developing Next-GenerationCost Models Jim Alstad, PhD Candidate USC Center for Systems and Software Engineering CS 510 Lecture September 17, 2014

2. Next-Gen Cost Models • Learning objectives for CS 510 lecture: • To provide a quick view of how an estimating model is developed • To describe some of the current research in cost model development

3. Next-Gen Cost Models Agenda Agenda: • Learning goals for 510 lecture • A brief history of the COCOMO family of estimating models • COCOMO III • COSYSMO 3.0 • COSATMO

4. COCOMO II and Some Derivatives • 1995: one-size-fits-all model for 21st century software • 1999: poor fit for schedule-optimized projects; CORADMO • 2000: poor fit for COTS-intensive projects: COCOTS • 2003: need model for product line investment: COPLIMO • 2003: poor fit for agile projects: Agile COCOMO II (partial) • 2012: poor fit for incremental development: COINCOMO

5. COCOMO Family of Cost Models Other Independent Estimation Models Software Cost Models DBA COCOMO 2004 COCOMO 81 1981 COCOMO II 2000 COCOTS 2000 COSYSMO 2005 COINCOMO 2004,2012 COSoSIMO 2007 iDAVE 2004 COPSEMO 1998 COSECMO 2004 COPLIMO 2003 COQUALMO 1998 AGILE C II 2003 COTIPMO 2011 COPROMO 1998 CORADMO 1999,2012 Software Extensions Legend: Model has been calibrated with historical project data and expert (Delphi) data Model is derived from COCOMO II Model has been calibrated with expert (Delphi) data Dates indicate the time that the first paper was published for the model

6. USC-CSSE Modeling Methodology - concurrency and feedback implied Determine Model Needs Step 1 Analyze existing literature Step 2 Perform Behavioral analyses Step 3 Define relative significance,data, ratings Step 4 Perform expert-judgment Delphi assessment, formulate a priori model Step 5 Gather project data Step 6 Determine Bayesian A-Posteriori model Step 7 Gather more data; refine model Step 8

7. Next-Gen Cost Models Agenda Agenda: • Learning goals for 510 lecture • A brief history of the COCOMO family of estimating models • COCOMO III • COSYSMO 3.0 • COSATMO Key slides provided by Barry Boehm, Vu Nguyen, and Brad Clark (CSSE 2014 Annual Research Review)

8. COCOMO II Data by 5-Year Periods Add in new data when calibrating COCOMO II Calibration

9. COCOMO II Data: Productivity Trends Calibrate to new productivity values COCOMO II Calibration

10. Application Experience (APEX)Rating Trends Very High is>= 6 years Reduce VH (etc) definition to cover same % of projects COCOMO II Calibration

11. Super-Domains and AFCAA Productivity Types

12. More on Application Domains • COCOMO III plans to identify a set of application domains, and identify and calibrate a submodel to each domain • Allows user to identify domain, and get a group of accordingly-set cost driver values off the shelf

13. COCOMO III Directions • CSSE’s April Annual Research Review discussions set these directions for COCOMO development: • 149 new data points have been added since the 161 data points used in COCOMO II.2000. • Add these data points • The average level of productivity appears to have increased • Therefore recalibrate the model’s parameters • The average level of several cost drivers appears to have changed • Consider changing the definitions of the high/medium/low levels for these cost drivers • More data points might add to variability in the data • Consider adding more cost drivers to improve the fit • Simplifying the model would make it easier to use • Define (calibrated) submodels for certain important application domains that provide off-the-shelf values for some cost drivers • Planned domains: Real-time, Engineering & Scientific, Command and Control, Automated Information Processing

14. Next-Gen Cost Models Agenda Agenda: • Learning goals for 510 lecture • A brief history of the COCOMO family of estimating models • COCOMO III • COSYSMO 3.0 • COSATMO

15. What Is Systems Engineering? • Some clauses of a definition: • Considers the system as a whole • Starts from stakeholder needs, results in a technical approach meeting those needs • Frequently develops system requirements from system needs • Technical approach is commonly stated as the system architecture • Systems engineers typically need to have familiarity with all the specific kinds of engineering involved in their type of system • Systems engineers can usually be found working on systems that are large, critical, and/or important • Participation in identifying all major risks and reducing/managing them (ICSM) • Typically most system engineering is done in the Valuation phase, with major work also being done in the Exploration and Foundation phases (ICSM) • See also ICSM book section 0.2.1 • The cost of system engineering needs an estimation model

16. History of COSYSMO Models COSYSMO 1.0Valerdi, 2005 Req’ts VolatilePena, 2012 With ReuseFortune, 2009 For ReuseWang et al, 2014 Sys of SysLane et al, 2014 COSYSMO 3.0Alstad, 2015? • Identifies form of model • Identifies basic cost drivers • Identifies Size measure • Adds scale factor based on requirements volatility • Adds weights to Size elements, reducing net Size in the presence of reuse • Adds weights to Size elements, reducing net Size when artifacts are only partially completed • Adds effort multiplier when in the presence of system-of-systems • Combines features of previous models

17. Basic COSYSMO (1/2) • COSYSMO [2] starts by computing the “size” of a system engineering project, in units of eReq (“equivalent nominal requirements”) • These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios. • Each artifact is evaluated as being easy, nominal, or difficult. • Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size:

18. Basic COSYSMO (2/2) • Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant. • This results in the following equation for a COSYSMO estimate of effort in person-months:

19. What Is Systems Engineeringfor Reuse? • Systems Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states: • Conceptualized for Reuse (e.g., Concept of Operations document) • Designed for Reuse (e.g., component detailed design) • Constructed for Reuse (e.g., integrated component) • Validated for Reuse (e.g., validated component)

20. COSYSMO For Reuse • A SEFR estimate adjusts each artifact’s size contribution by considering its SEFR state according to this table:

21. What Is Systems Engineeringwith Reuse? • Systems Engineering with Reuse is project development, with reusable artifacts being brought into the product • A special case: zero reusable artifacts • Each reusable artifact is included in one of these SEWR states of maturity: • New (i.e., not reused) • Re-implemented (through requirements & architecture) • Adapted (through detailed design) • Adopted (through implementation) • Managed (through system verification & validation)

22. COSYSMO with Reuse • A SEWR estimate adjusts each artifact’s size contribution by considering its SEWR state according to this table:

23. Next-Gen Cost Models Agenda Agenda: • Learning goals for 510 lecture • A brief history of the COCOMO family of estimating models • COCOMO III • COSYSMO 3.0 • COSATMO

24. The Problem • How much will the total system cost? • Is one phase being optimized while increasing total cost? • Is the system affordable? • Does the acquisition comply with the Better Buying Power intiatives (DoD)? $?

25. The Solution Example acquisition process (DoDI 5000.02) COSATMO assists acquirers and developers during these phases (highest payoff during early phases) COSATMO estimates the cost for these phases

26. COSATMO Objective • Context: • Current and future trends create challenges for full-system cost estimation • Emergent requirements, rapid change, net-centric systems of systems, COTS, clouds, apps, widgets, high assurance with agility, multi-mission systems • Current development practices can minimize cost of one phase, such as development, while raising full-system cost • The COSATMO project is developing a modern full-system cost model (first space systems, then other DoD domains) • “Constructive SATellite cost MOdel” • Current estimating models focus on one aspect, such as system engineering • COSATMO will enable: • System-level trades to be handled within a single model • Easy customer evaluation of full-system cost • Modern technologies to be covered

27. Segments of Satellite System Cost • Total satellite system cost [tied to slide 25 phases] = System engineering cost [EMD] • Satellite software cost [EMD] • Satellite vehicle hardware development [EMD] and production [Prod] cost • Launch cost [Deploy] • Initial ground software cost [EMD] • Initial ground custom equipment cost [EMD] • Initial ground facility (buildings, communications, computers, COTS software) cost [EMD] • Operation & support cost [Deploy, O&S] • Updated at GSAW (Feb 2014) • Model as sum of submodels is new structure in COCOMO family

28. COSATMO Segment Tentative Models • System engineering: COSYSMO, perhaps with add-ons • Satellite vehicle hardware development and production: Current Aerospace hardware cost model(s); exploring extensions of COSYSMO for hardware cost estimation • Satellite vehicle, ground system software development: COCOMO II, COCOTS, perhaps with add-ons • Launch model: similarity model, based on vehicle mass, size, orbit • Ground system equipment, supplies: construction, unit-cost, services cost models • Operation & support: labor-grade-based cost models, software maintenance models

29. Key Overall Satellite SystemCost Drivers • Most Important: • Complexity, Architecture Understanding, Mass, Payload TRL level/Technology Risk, and Requirements Understanding. • Important: • Reliability, Pointing Accuracy, Number of Deployables, Number of Key Sponsors, Data Rate, and Security Requirements for Communications. • Determined at COCOMO Forum (Oct 2013)

30. Ground System Segment Development (1/2) • Determined at GSAW (Feb 2014) • Ground system-wide cost drivers • Most important: Accreditation (information assurance, etc), Required security • Also important: # satellites* • Initial software cost drivers • Required data throughput • Generally handled by COCOMO II, COCOTS, COPLIMO • *Indicates a size measure

31. Ground System Segment Development (2/2) • Ground custom equipment cost drivers • Most important: Amount of new development required, # of custom equipment sites*, Required site availability & reliability, Required site security • Also important: # driving requirements* • Ground facility cost drivers • Most important: # facilities*, location of facilities (especially US vs foreign), # ground RF terminals* • Also important: Facility “reuse” • Operation and support cost drivers • Most important: # years of operation*, # FTE staff (with labor mix)* • Also important: Size of software maintained*, Leased line cost*, level of automation • *Indicates a size measure

32. COSATMO as a Research Umbrella • General direction: • Develop a full-coverage satellite system cost estimating model • Generalize that to additional applications Specific current research initiatives: • COSYSMO 3.0 • COCOMO III Research vehicles: • My thesis • Other theses • Other research

33. Bibliography • “A Generalized Systems Engineering Reuse Framework and its Cost Estimating Relationship”, Gan Wang, Garry J Roedler, Mauricio Pena, and Ricardo Valerdi, submitted for publication. • “The Constructive Systems Engineering Cost Model (COSYSMO)”, Ricardo Valerdi (PhD Dissertation), 2005. • “Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0)”, Jared Fortune (PhD Dissertation), 2009. • “Quantifying the Impact of Requirements Volatility on Systems Engineering Effort”, Mauricio Pena (PhD Dissertation), 2012. • “Life Cycle Cost Modeling and Risk Assessment for 21st Century Enterprises”, Barry Boehm, Jo Ann Lane, Supannika Koolmanojwong, Richard Turner (presentation), April 29, 2014. • "System Interoperability Influence on System of Systems Engineering Effort", Jo Ann Lane, Ricardo Valerdi, unpublished. • “COSYSMO Extension as a Proxy Systems Cost Estimation” (presentation), Reggie Cole, Garry Roedler, October 23, 2013.