How Big is Yours? The use of Comparator tools in Estimation

1. How Big is Yours?The use of Comparator tools in Estimation Andy Nolan BSc Hons, CEng, FBCS, CITP Chief of Software improvement – The Software Centre of Excellence Satpaul Sall BSc Hons Software Technologist – The Software Centre of Excellence

3. A Brief History of Engine Controls 3 • The Control Systems department is responsible for the Engine Electronic Controllers (EECs) for a range of small and large gas turbine engines for the aerospace industry. • The software is developed to DO-178B Level-A standards • The company has been developing high integrity software for over 20 years and has extensive data on its processes and productivity. • We have the largest order book in history, new engine development places greater demand on the software team (shorter time scales and lower costs)

4. CO-Bus-MO Enterprise business performance Software Supplier COCOMO Challenging our supplier costs. Benchmarking the business Hardware COCOMO CO-RISK-MO Risk Management Using the model to identify and quantify risk Using many SW factors to estimate hardware engineering CO-Imp-MO Improvement COCOMO Hardware Supplier COCOMO Using COCOMO to identify & validate improvements Estimation & eliciting key assumptions & negotiation COCOMO A unifying language

5. The Comparator Tool

6. Assumption The engineering practice of translating requirements into a implementation will be “similar” in many engineering domains and will be subject to the same cost drives as software

7. Cost = C * Size * Environment Size (the product): The magnitude (or quantity) of a task. • Size • Complexity • Reuse • Risk/Uncertainty Environment: The environment in which you build the product • Processes & tools • People • Management • Organisation • Etc Higher Cost Environment Lower Cost Size

8. Historic Baseline New Project Historic Size Delta Create a scaling Factor that represents the differences between the two projects in both size and environment New Size Historic Environment (a definition of the project environment) New Environment (a definition of the project environment) Actual historic cost New cost = Historic cost * Delta Comparator Tool

9. Examples

10. Absolute Tool An “Absolute” estimation tool was developed for Hardware by relating hardware complexity to software lines of code. This relationship is hidden from the estimator who only has to select hardware units they are working on New Project Standard job hours Size (task and unit) Environment (a definition of the project environment) cost = Standard Job * Size * Environment

11. Calibrating a Whole Engine R&D Cost Model

12. Abstract • Business Challenge • Can we use COCOMO to estimate the cost of a whole engine? • Effort • We have 8 hours to demonstrate this capability! • Approach • Use Comparator method based on COCOMO II • Relate all aspects to a nominal “Baseline” project • COCOMO II factors to be completed by Chief Design Engineer • No underlying factors to be altered – only the constant “C” • Leave 1 project aside as a test of calibration (project 6)

13. Environment: Relative (COCOMO II factors) Engine 1 Engine 2 Engine 3 Engine 4 Engine 5 Engine 6 Low requirements volatility, high team cohesion, high precedence, mature processes, experienced team, low schedule pressure, experienced management team Stable requirements, higher precedence, team more experienced, single site team, improved process maturity., high levels of reuse Stable requirements, higher precedence, single site team, improved process maturity. Improved management Lower requirements change, team cohesion, process maturity, experienced team The only show in town – high priority. Single site project, stable team High process maturity, High requirements volatility, low TRL, low process maturity, low team experience, low management experience and high schedule pressure High Schedule pressure Some architectural issues. Drop in team experience, high turnover of staff, multi site working Requirements change, loss of precedence, inexperienced team, architectural issues, loss of team cohesion, schedule pressure, new management team, multi site project, staff turnover Loss of precedence, some architectural issues, schedule pressure, multi site project, high staff turnover

14. What correlated well • Size • REVL • PREC • FLEX • RESL • TEAM • PMAT • SCED • Management Expereince • Based on the survey, these factors correlated well with cost. This does not mean the other factors do not – rather it means that these were the factors the interviewee understood well.

15. Predicted & Actual costs – A close correlation Calibrate to these projects Estimate this project Baseline

16. Cost = C * Size * Environment 1 4 2 Environment (not to scale) 5 B 3 6 Size (AMF) – not to scale

17. Conclusions • Size is not the only determining factor for cost – the environment is also proving critical for project success. • From a simple analysis, it would seem that COCOMO II can be used to “model” the development environment • The Rolls-Royce Environment is changing and COCOMO II predicts that this is creating a headwind • The exercise has the side effect of training the business leaders in the factors that affect cost

18. Why Estimation Tools Help the Business

19. COCOMO II: A common Language • COCOMO II is a language that bridges between engineering and the business • It is also the bridge between different areas of the business • It has been used to help benchmark the business

20. Remain within a domain • You can interpolate and extrapolate within a domain but not across domains (unless you have a domain bridge function) • COCOMO II appears to work on engineering practices where requirements are translated into implementation through an engineering process