2nd Annual ICEAA Canada Workshop 22-23 February 2016 Rick Collins

1. A Look Inside a High Performing Cost Analysis Organization 2nd Annual ICEAA Canada Workshop 22-23 February 2016 Rick Collins CEO, Technomics, Inc. rcollins@technomics.net 571-366-1402 (Office) 571-218-0411 (Mobile)

2. Outline • Some Critical Context • Distinguishing Features • Potential Capability Maturity Model • Closing Food for Thought

3. You Can’t Manage What You Don’t Understand Credible Cost Estimates Realistic Budgets Executable Contracts/Projects Decision-makers demand & rely on advice from high performing cost organizations Must avoid starting projects that can’t be finished Improved Acquisition Outcomes The right cost estimating capability facilitates authoritative decision-maker knowledge that enables informed decision making

4. The Big Picture Is one of these ingredients most important?

5. Distinguishing Features – People (1 of 2) • Staff willing & able to think • Staff willing & able to think independently • Staff who ask the right questions • Staff willing & able to convey & defend ‘bad’ news • Staff who can anticipate the next question • Staff who recognize that there is generally more than one way to solve a problem & make the time to do so • Staff who won’t rest on their or others’ laurels • Staff willing & able to perform cost research leading to improved data and techniques/tools • Staff who can see the big picture and will proactively act on that perspective • Staff who understand that they have unique knowledge and value that extends beyond cost and will proactively act on that knowledge “A fool with a tool is still a fool.”

6. Distinguishing Features – People (2 of 2) • Leaders create the right culture to attract, develop, empower, reward and retain talent that exhibits the aforementioned behaviors • Leaders understand continual investment in data, techniques & tools is essential to analytical credibility • Leaders understand the manner in which results are presented is more important than what’s presented • Leaders understand the ‘right’ analysis at the right time is invaluable; great analysis at the wrong time is useless • Leaders recognize certifications are not a substitute for on-the-job experience practicing the science and art of cost analysis • Leaders recognize importance of rigorous (though not onerous) quality control processes • Leaders ensure their bosses understand organizational value well enough to enable organizational health via marketing, resources, etc. Leaders orchestrate the optimal investment in & deployment of their assets

7. Distinguishing Features - Processes Cost Estimating Process Cost Assessment Process Other Processes • Data collection & normalization • Cost estimating relationship development • MS Excel cost model development • Risk and uncertainty analysis • Quality control • … • … Standard processes = repeatability, transparency, accuracy & credibility

8. Distinguishing Features - Data • Authoritative data is readily accessible, well organized & relied upon • Authoritative data is maintained to ensure recency, completeness & efficiency Raw and normalized actuals Quantity, schedule & other contextual info Physical and performance parameters High performing cost orgs do what it takes to get the right data

9. Distinguishing Features – Techniques/Tools • Analytical and visualization methods that enable analytical results ensuing actionable insights • Methods are maintained/enhanced/developed to ensure relevance WBS/cost element-specific algorithms Automated compilation of CERs Data graphics that enable more effective analysis & storytelling High performing cost orgs invest in developing & acquiring the right enablers

10. Capability Level Entry Basic Mature Technical Baselines ** ü Quality of Data ** ü Estimating Techniques ** ü Automated Tools ü Repeatable Processes ü Capability Maturity Model Enabled Outcomes Mature Delivery On Time/At Cost Basic Executable Contracts Insightful Performance Measurement Entry Realistic Budgets Maturing Capability Informed Assessments & Negotiations Attribute Credible Cost Estimates Cost Analysis Support Continuum Monitoring Evaluation Planning ** Definitions of each capability level follow Time High performing orgs routinely look in the mirror, learn and evolve

11. Maturity Levels for Realistic Technical Baselines • Realistic technical baselines provide the necessary inputs for developing a credible cost estimate. Inaccurate and/or missing key technical data typically results in cost estimates based on optimistic assumptions, aka good intentions. • Entry – Technical data elements are collected on ad-hoc basis with no standardization across projects. • Basic – Technical data elements are collected on an ad-hoc basis with standardization across projects. • Mature – Technical data elements are collected and reported in a standard process across programs/projects High performing cost orgs enable technical baseline realism

12. Maturity Levels for Quality of Data • High quality historical cost, technical and programmatic data is required to develop credible estimates. Quality of data includes many variables such as: granularity (level of detail), volume (number of related projects), relevance (applicability to future projects) and validity/accessibility (each us dependent on having a formal data collection process and system). • Entry – Limited high-level data exist for primarily foreign nation projects. Limited number of relevant Canadian experiences available. The number of actual projects is few and not completely analogous for application. Ad-hoc data collection techniques and storage methods exist. • Basic – More detailed data exist. Small number of Canadian experiences available to analyze and represent some diversity of ship types. Application of data still requires significant extrapolation. Standardized data storage and sharing processes have been defined to support info access and sharing. • Mature – Detailed data exists for many foreign and multiple Canadian projects that represent a healthy population of analogous ship types. A data collection process and system exists with well-defined data collection processes, requirements, validation techniques and web-based sharing. High performing cost orgs collect the right data

13. Maturity Levels for Estimating Techniques • Valid analytical estimating techniques that accurately represent the data and potential explanatory variables facilitate the credibility of estimates. They also provide the foundation for developing homegrown estimating tools or populating/calibrating commercial estimating tools. • Entry – Existing estimating techniques are used as-is, despite weaknesses that warrant improvement. • Basic – Cost research performed on as-needed basis produces new or updates existing defensible estimating techniques. Standard estimating techniques for some significant elements and phases of cost are available to cost analysts. • Mature – Cost research performed on a routine basis produces new or updates existing defensible estimating techniques. Standard estimating techniques for all significant elements and phases of cost are available to cost analysts High performing cost orgs exploit the right data

14. Closing Food for Thought • High performing cost organizations invest to achieve the right balance of the four ingredients • High performing cost organizations understand where/how they are delivering value to the enterprise and leverage this performance information to their advantage