Lecture 4.3a: Metrics Overview (SEF Ch 14)

1. Lecture 4.3a: Metrics Overview(SEF Ch 14) Dr. John MacCarthy UMBC CMSC 615 Fall, 2006

2. (Program) Metrics in Management • (Program) Metrics are (quantitative) measurements collected for the purpose of determining project progress and overall condition • Metrics are used to manage risk (technical, cost, and schedule) • Metrics are collected and reported periodically (to management and the customer) • Metrics are measured over time so that changes (trends) can be observed • Technical Management requires three types of Program Metrics: • Product Metrics: track the development of the product • Earned Value: tracks conformance to the planned schedule and costs • Management Process Metrics: track management activities • DoD Policy requires the identification and tracking of TPMs

3. Product Metrics: Demonstrate progress toward meeting customer requirements Three Types: Operational Performance Life-cycle Suitability Affordability Technical Performance Measurements (TPMs): A set of (5-20) key product performance metrics selected for tracking and periodic reporting to management. Derived from MOPs Generally allocated to CIs Metric Hierarchy: Measures of Effectiveness (MOEs): high-level measures of a system’s ability to achieve operational mission success (See CDD KPPs) Measures of Suitability (MOSs): high-level measures of the extent to which the system integrates will into the operational environment. They generally include supportability, human interface compatibility, and maintainability Measures of Performance (MOPs): high-level physical or functional attributes of the system that relate (trace) to the MOEs and MOSs. Product Metrics [1]

4. Suitability Metrics: Degree to which a system can be placed satisfactorily in field use: Availability, Reliability, Maintainability (RAM) Usage Rates Supportability & Logistics Human Factors, Manpower, Training, & Safety Interoperability, compatibility, transportability Environmental Impact Robustness, Survivability Producibility, Testability, Simplicity … TPMs often include suitability metrics MTBF MTTR Affordability Metrics: LCC drivers: (Development Costs) Production Costs Operational Costs Maintenance Costs Disposal Costs Product Metrics [2]

5. Earned Value (14.2) • Earned Value is the standard metric for tracking cost and schedule progress and risk • See Separate Briefing on Earned Value

6. Process Metrics (14.3) • Management Process Metrics: measurements taken to track process of developing, building and introducing the systems: • Examples include: • Availability of resources (personnel) • Number of new hires • Items Completed • Customer Satisfaction • Trouble Tickets • Lines of Code (developed) • Complexity • ECPs baselined • Time to approve documents • Number of High Risks • Etc. • These are important for CMMI

7. Summary Points (14.4) • Technical Management requires three types of metrics • Product Metrics • Earned Value • Management Process Metrics • Metrics should be reported on a periodic basis • The measurement, evaluation, control, and reporting of metrics should be planned to assure proper measurement, evaluation and dissemination • TPMs are performance-based product metrics (key technical parameters) that are used to track progress toward meeting the program’s operational requirements • Use of TPMs is required by DoDI 5000.2

8. Technical Performance Measurement (S14-A) [1] • See separate TPM briefing