CLR 252 Developing KPPs

1. CLR 252Developing KPPs Note from SME: Changes to text are marked in Red. If just the title is in red, that is all that is changed—If the title AND text (or partial) changed—it will be marked. EXCEPTION—If there is a new DIAGRAM—I only marked the title/header—to not “mess up” the diagram.

2. Lesson #3: How to Write a Good Key Performance Parameter Components and Guidelines for writing

3. Lesson Objectives • Terminal Learning Objective • Describe the elements of a good requirement Enabling Learning Objectives • Identify components of a well written KPP • Apply the Guidelines for writing quality KPPs • Explain the relationships of the KPPs and KSAs to Measures Effectiveness (MOEs), Measures of Performance (MOPs), and Measures of Suitability (MOSs)

4. Lesson Overview • Elements of a KPP • Guidelines for Writing Quality KPP • Common Characteristics of Well-Written KPP • Measures of Effectiveness • Measures of Performance • Measures of Suitability

5. Elements of a KPP Each KPP should be: • Achievable • Measurable/Testable • Unambiguous • Complete • Necessary • Prioritized We will look at each of these elements in more detail…..

6. Elements of a KPP • Achievable – It must reflect need or objective for which a solution is technically achievable at costs considered affordable. • Measurable/Testable – Each KPP must be testable by such as, inspection, demonstrate, analysis or test. • Unambiguous – It must have only one meaning. Each KPP should be concise and simple. Multiple readers of a requirement should arrive a the same interpretation. A requirement should be simple not compound, that is, each requirement should have a single purpose.

7. Elements of a KPP • Complete – All information should be listed for the specific KPP • Necessary – Each KPP should document something the customer/user really need or something that is required for conformance to an external requirement, an external interface, or a standard. Trace each requirement back to its origin. • Prioritized – Not all KPPs are created equal, and not every KPP can be the number one issue. Priority is a function of the value provided to the customer, the relative cost and the risk.

8. Elements of a KPP Check for… If not… • Traceable to a need/objective 1. Reject • Consistent with other requirements 2. Reject • Not redundant to other requirements 3. Reject • Clearly stated 4. Rewrite • Feasible (current state-of-the-art) 5. Reject • Defines single objective 6. Decompose • Measurable 7. Redefine • Unambiguous 8. Clarify • Complete definition of all objectives 9. Continue

9. Knowledge Review _______is a function of the value provided to the customer, the relative cost and the risk. a. A standard b. A requirement c. Priority d. A customer need

10. Guidelines for Writing Quality KPPs Now that we have looked at basic elements of a KPP, let’s explore some time-tested guidelines for writing quality KPPs.

11. Guidelines for Writing Quality KPPs • Keep sentences and paragraphs short, use active voice, and use proper grammar, spelling, and punctuation • Develop the KPP from the user’s point of view • Develop KPPs that are derived from the ICD • Ask yourself “WHY” and “WHAT” questions and not HOW • Use words that are directive in nature • Don’t be over stringent

12. Guidelines for Writing Quality KPPs • Use tolerance ( ≤, ≥, range, ±); helpful to the designer • Write testable KPPs • Watch out for multiple KPPs • Never use and/or in a KPP • Use common sense

13. Reflective Question/Knowledge Review How does writing a KPP with a set tolerance (+/-; range) help a designer? Which words below are directive in nature. Check all that apply. a. Shall b. Will c. Should d. Must e. May

14. Common Characteristics of Well Written Requirements Wayne Turk’s article from Defense AT&L: September-October 2005 Mission Possible … With Good Requirements provides a well though-out list of Common Characteristics of Well Written Requirements.

15. Common Characteristics of Well Written Requirements They must be: Measurable Attainable Necessary Correct Unambiguous Orderly Organized Results-Oriented Source: Defense AT&L: September-October 2005 Mission Possible … With Good Requirements Wayne Turk

16. Common Characteristics of Well Written Requirements Good Requirements need to be Quantifiable Measurable Verifiable/Testable How to verify? Developmental Test and Evaluation Operational Test and Evaluation

17. Common Characteristics of Well Written Requirements Good Requirements need to be: Feasible Attainable Achievable What does that mean? • Can it be built within current technology possibility • Within available time • Within available money

18. Common Characteristics of Well Written Requirements Must be necessary – threshold Is it necessary to accomplish the mission? We have fiscal constraints No room for: • Nice to have – objective values – no extra cost** • Desires • Frivolous

19. Common Characteristics of Well Written Requirements Must be unambiguous Multiple readers must= the same understanding The Interpretation of requirement is not a good thing Don’t use: • User-friendly • Fast • Easy • Flexible • State-of-the-art • Maximize/minimize • Efficient • Semi-automatic

20. Common Characteristics of Well Written Requirements Must be orderly Prioritize requirements • Use KPP, KSA and Attribute tables There are constraints • PM may trade-off all requirements except for KPPs • Priorities will help PM decisions • PM may trade within the threshold-objective range of each KPP

21. Common Characteristics of Well Written Requirements Must be organized • Strategy to task • Trace thru Identification to Development to Verification Group into categories (JCA) This cuts down on: • Duplication • Inconsistencies • Contradictions

22. Common Characteristics of Well Written KPPs Must be results-oriented What does the system have to DO? • Capability based • What are the needs • Not how to accomplish

23. Potential KPP-Writing Traps Now that we have looked at guidelines for writing quality KPPs and common characteristics of well-written KPPs, we have to look at some potential problems with requirements writing and how to avoid them.

24. Potential Requirements-Writing Traps • Poor Word Choice leads to ambiguity • Say what you mean • Rambling requirements can cause confusion • Watch out for this in technical specifications or interface requirements Source: Defense AT&L: September-October 2005 Mission Possible … With Good Requirements Wayne Turk

25. Potential Requirements-Writing Traps • Being too Specific • Trying to design within the requirement– can stifle innovation • Trying to add requirements not backed up by analysis • Unrealistic expectations • No system is 100% reliable or completely upgradable

26. Basic Rules to Avoid Potential Requirements-Writing Traps • Keep Users involved • Develop and refine requirements • Define and use consistent terminology • Organize requirements by JCA

27. Remedies to Avoid Potential Requirements-Writing Problems • Monitor and track development changes** • Document all requirements and changes and why they changed ** • Follow the JCIDS/DAS process ** pop-up for article of teaching point negative example in notes page

28. Reflective Question/Knowledge Review For a requirement to be feasible, attainable and achievable, What does it have to be able to do? Check all that apply. a. Built with today’s technology b. Built within available time c. Built within available money d. Built to original specifications

29. Measures of Effectiveness and KPPs Now that we have looked at components and guidelines for writing quality KPPs, we will look at the link between KPPs and measures of effectiveness.

30. Measures of Effectiveness and KPPs Measure of Effectiveness (MOE) Measure designed to correspond to accomplishment of mission objectives and achievement of desired results. (DAG, Ch 9)) MOEs may be further decomposed into Measures of Performance and Measures of Suitability. What do we want the system to be able to do? Link to DAG and DOT&E memo

31. Measures of Performance and KPPs Measure of Performance (MOP) Measure of a system’s performance expressed as speed, payload, range, time on station, frequency, or other distinctly quantifiable performance features. Several MOPs and/or Measures of Suitability may be related to the achievement of a particular Measure of Effectiveness (MOE). What operational capability does this system provide? - Mission related KPPs

32. Measures of Suitability and KPPs Measure of Suitability (MOS) Measure of an item’s ability to be supported in its intended operational environment. MOSs typically relate to readiness or operational availability, and hence reliability, maintainability, and the item’s support structure. Several MOSs and/or Measures of Performance may be related to the achievement of a particular Measure of Effectiveness (MOE). How reliable is the system? - Sustainment related KPPs *pop-up : not all services use MOS as a term

33. Measures of Performance and KPPs - JSF Example Measure of Performance KPPs - Combat Radius - 3 Profiles by Service: USMC 450nm; USAF 590nm; USN 600nm - Radio Frequency Signature - (Stealth) - Interoperability – Meet 100% of critical Information exchange requirements: Secure Voice and Data

34. Measures of Effectiveness and KPPs - JSF Example

35. Measures of Suitability and KPPs - JSF Example Measures of Effectiveness (MOE) Measure of Suitability KPPs - Mission reliability directly impacts O&S costs for parts replacement and the associated expenditure of maintenance man-hours.   - Logistics footprint influences both program acquisition costs and O&S costs; the smaller the footprint is, the smaller the acquisition cost will be and the less expensive it will be to transport and maintain. - Sortie generation rate depends on maintenance man-hours per operating hour and heavily influences design for maintainability

36. Reflective Questions/Knowledge Review What KPPs are more important to your programs – Measures of Performance or Measures of Suitability? How would you convey your priorities to your Acquisition counterpart? Which of the following metrics would be a measure of performance and which would be a measure of suitability? Unrefueled Range 150-200 nm Transportability on a C-130 Combat ready within 1 hour of arrival Able to carry 8 fully combat loaded troops Travel 35 mph with full fuel/troop load on and improved road

37. Lesson Review • Elements of KPPs • Guidelines for Writing Quality Requirements • Common Characteristics of Well-Written Requirements • Measures of Effectiveness • Measures of Performance • Measures of Suitability

38. Lesson #3 Which of the following are descriptive words a requirements manager would want to use to write a valid requirement a. No less than b. User-friendly c. Fast d. Easy e. Flexible If a requirement is not clearly stated, the requirements manager should: a. Reject b. Redefine c. Clarify d. Rewrite

39. Lesson #3 ELO # 2 Using the phrase “state of the art” in a requirement referring to technology will ensure allowance for future upgrades. a. True b. False A Range Tolerance will help the program manager and designers. a. True b. False

40. Lesson #3 When writing the requirement, do not use except, if, when, unless and but because they: a. Give a range for the contractor to work b. Provide escape clauses c. Prevent exceeding thresholds d. Help with testing Using and/or helps designers and program managers with trade space. a. True b. False

41. Lesson Test ELO #3 Measures of Suitability typically relate to: a. readiness or operational availability b. reliability c. maintainability d. the item’s support structure What measure is designed to correspond to accomplishment of mission objectives and achievement of desired results? a. Measure of Performance b. Measure of Suitability c. Measure of Effectiveness d. Measure of Reliability

42. Lesson Test ELO #3 A new light cargo aircraft has a KPP of being able to take off on a 3000 foot dirt runway with 15k of cargo. The KPP would be a: a. Measure of Suitability b. Measure of Performance c. Measure of Reliability d.Sustainment KPP The airlifter has a KPP that states it must be self-deployable (must be able to haul its own maintenance package) The KPP would be: a. Measure of Performance b. Measure of Suitability c. Measure of Effectiveness d. Measure of Reliability