Data for Costing & Cost Uncertainty Mort Anvari Director, Acquisition Costing (SAFM-CEA)

1. Data for Costing & Cost Uncertainty Mort Anvari Director, Acquisition Costing (SAFM-CEA) June 18, 2008 Cost Management Panel PRICE Customer Symposium

2. Key Points • We have Several Data Quality Issues and Potential for Type I (α) and II (β) Errors in our Cost Estimating. • Opportunity for Cost Analysis Enterprise Resource Planning (CA-ERP) and Data for Costing Architecture. • Need for Range Cost Estimating and Cost Confidence Analysis Need Data for Costing: Future Technologies; Processes; Performances; Products; Services; Personnel; and Schedule.

3. Army Cost Data Sources Army Cost Positions (ACP) Database ACE-IT Sessions, Periodical SAR, DAES, WSR, APB, P-Form, Contracts Summaries +

4. Performance Cost Schedule Data for Cost Estimating Information Knowledge Data Cost Models • Historical Data: • Cost and (Physical, Technical, Performance, Capability, Schedule). • Subject System Data: • Physical, Technical, Performance, Capability, Schedule. Estimates are only as good as the data that they are based on.

5. Data Issues • Current Data Reporting and Usage Issues (CSDR CCDR, SRDR, and CPR). • Family or System of Systems (Commonality Issue) • LSI, JV, Prime, Sub (Contracting Issue) • CPIF, FFP,. . Reporting Requirements (Contract Type Issue) • Manufacturing, Services, SW, and ERP (Activity Type Issue) • Mod, ECP, and Incremental Contracts (Completeness Issue) • Competition and Business Base Issues (Quantity issue) • Physical, Technical, TRL, Performance (Technology Issue) KPP and Capability Data Issue Allocation Issue: Mapping to Standard Reports Labor – Material Direct – Indirect Recurring – Non-Recurring

6. Data Quality Challenges • Lack of data. • Physical, technical, performance, capability. • S/W ERP FFP Contract. • ACAT II and below programs. • Policies and procedures. • DoDI 5000-2 dated May 2003. • Requires SRDR (w/o Cost) but not CCDR for some programs. • Vague in some areas; CSDR “not required for procurement of commercial systems, or for non-commercial systems bought under competitively awarded, firm fixed-price contracts, as long as competitive conditions continue to exist”. • Army regulations do not require additional cost reporting. • Data collection process. • Existing data is not readily available to the Cost Analysts; sometimes they are not even aware that data exists. (Data in Jail) • Contractors required to map CSDR reports IAW DoD MIL-HDBK-881.

7. Data Quality? CSDR Data Collection Process Labor vs. Material Direct vs. Indirect Rec. vs. Non Rec. Unique vs. Common Contractor allocation Contracting Contractor Data / Reports: Contractor1 Contractor Accounting Systems: G2 R3 Contractor 2 Contractor Allocation R12 R8 R7 R5 R18 S4 R16 R1 R11 R14 S2 R2 Contractor3 S5 R4 S6 R10 S1 G1 G3 S9 R17 R13 S7 R15 S10 R9 S3 S8 R6 Process vs. Quality. • Process affects data quality. • Increases probability for data errors. • Results in mapping / allocation inconsistencies.

8. Cost Data Flow Contractor Cost Accounting Systems Contractor Sys3 Contractor Sys4 Contractor Sys2 Contractor Sys1 Government Informational Systems / Databases OSMIS CLS ACDB Cost Analysts Current Information Access Flow. • Need for Data Warehousing. • Repository of electronically stored data. • Several databases typically supply data to the warehouse. • Improved ease of data retrieval and data quality.

9. Data Warehousing ExternalData Contractor Cost Accounting Systems Information Delivery System DataWarehouse Government Informational Systems / Databases Data Marts External Users Proposed Information Access Flow. • Raw Data loaded directly from contractor accounting system into data warehouse. • Metadata: ‘Data about Data’. • Data visualization / data warehousing / intelligent data mining. • Global definitions that can be referenced by many different databases.

10. Cost Analysis ERP • Potential for Enterprise Resource Planning (ERP) of Cost Analysis Data and Models across the DoD. • Enable data and information sharing. • Allow automation and integration of business processes. • Data for Costing – Vision: • Collect raw data with more flexibility and focus on metadata development – XML. • Data warehousing of raw and current data. • Acquiring data mining and search tools to assist data analysis. Cost Analysts need to be more involved in the process. Data ERP for Costing? Check out http://www.anvari.net/E_Proc_Model.htm

11. Cost Risk Analysis Risk analysis reduces the uncertainty between requirements and funding

12. 100% Confidence 75% Cost $ 50% 25% 0% Causes of Cost Variations • Technical & Schedule Risk (Included) • Cost Estimating Methodology Uncertainty (to-be Included) • Engineering • Analogy • Parametric • Physical Parameters • Technical Parameters • Performance Parameters • Capability Parameters • Expert Opinion • Programmatic Changes (Mod) (Excluded) • Restructuring • Reprogramming • Rescheduling • Resourcing Cost Confidence Curve or “S” Curve

13. Current & Proposed Estimating Methods • Current: • Point Estimate Costing and Reporting, with Additional Cost Range Modeling Option at Technical & Schedule Risk Items. • Narrow Band Confidence. • Utilities are Risk Management and Execution. • Proposed: • Range Estimate Costing and Reporting, with Point Estimate Reporting Option at Different Confidence Level. • Wide Band Confidence. • Utilities are Planning and Programming.

14. 100% 75% All Risks and Uncertainties Technical Risk 50% Confidence Level 25% 0% Cost $ Cumulative Distribution Functions Increasing cost confidence from 50% to 75% Path Forward: Increase cost estimating confidence and implement rigorous risk analysis

15. Backups Backups

16. Surface Vehicles Missiles / Munitions Com / Electronics Aircraft Current Cost Data Collection Efforts • Automated Cost Databases (ACDB) • Primary source of data is CSDRs, CPRs, etc. • Includes programmatic and technical information. • Cost Analysis Requirements Description (CARD) • Redesigned to reduce variation (IAW Lean Six Sigma process). • On-line living document. • Standardized format for documents. • Controlled author access to CARD sections. • Milestone feeder documents (draft) provided early. • New process allows variation to be controlled (man-hour avoidance). • Capability Knowledge Base (CKB) • Performance/capability-based relationships. • Uses capabilities and costs of current systems to provide rough order of magnitude estimates for capability-filling solutions. • Cost support for earlier acquisition decisions.

17. Current Cost Data Collection Efforts • Operating and Support Management Information System (OSMIS) • Used by over 1,400 DoD employees (government and contract). • Over 25,000 queries generated annually. • Data used to generate OPTEMPO Weapon System Factors; direct link to PPBE process. • Historical archive of O&S data for period covering 1993 to 2007. • Prior year data is available by 1 April of the following year; Monthly activity reports by serial number level are available by 25th of the following month. • Contractor Logistic Support (CLS) Data • CLS accounting for more of the total OPTEMPO requirement. • Established a standardized process for CLS data collection. • Requirement place on contract in conjunction with CSDR requirements. • Cost data will enable the Army to forecast more reliable cost estimates for supply support.

18. Current Cost Data Collection Efforts • Joint Integrated Analysis Tool (JIAT) • Cost estimating, engineering design, requirements, capability and performance analysis linked. • Ready access to cost and technical data in each commodity area: Missiles, Aircraft, Vehicles and Communications and Electronics Systems. • Supports cost as an independent variable (CAIV) analysis, capabilities costing and trade-off analysis. • Near real-time cost estimating capability to the acquisition and requirements communities. • Software (SW) Data Collection Initiatives • Collecting SW development actuals on past programs. • Emphasis on improving SW maintenance estimates . . . Working with Army SW Engineering Centers collecting SW maintenance actuals. • Partnering with Air Force & Navy . . . Collect & share data on Enterprise Resource Planning (ERP) Projects and SW intensive programs.