Risk Based Quality

1. Risk Based Quality July 17, 2001 Quality Leadership Forum Tom Whitmeyer Manager, Agency Quality Program (1)

2. What is the Risk Based Quality Approach? Increasing Emphasis or Levels of Surveillance in Specific Areas 100% 0% Involvement Surveillance where needed, when needed, to the degree needed All or nothing Quality Finance Property Coordination Engineering Procurement Discipline Unique Integrated Planning High Risk More Definition Low Risk Add on, after the fact Built into contract/project Implementation Static, “cut-paste’, “us-them” Active, proportional,” partnering” Communication File Report Results Analyze Re-Deploy Stand alone -self sustaining Supporting project/contract management

3. Based on Government “Requirements” Emphasis on “How” Sometimes “Cut and Paste” Somewhat “Static” Quality Compliance Government Requirements and Industry Consensus Practices Tailored to Unique Risk Some “How”, and Additional Emphasis on “What” and “Why” Based On Risk More Requirements Tailored to Risk More Evolution as Risk Changes Quality Compliance and Additional Focus on Defining and Managing Quality Related Risk What are Areas of Increasing Emphasis for Risk Based Quality? Increasing Emphasis

4. Build Test Operate Maintain What is the NASA Risk Based Quality Strategy? Policy Risk Based Surveillance Supplier Assessment Quality Problem Reporting and Corrective Action EEE Parts Process Control Understand “Why” and “How” Ensure knowledge is captured, integrated, and utilized continuously to improve quality PBMA, Pictorial Reference Book, Technical Reports, Web-sites, Application Notes, Points of Contact, Contract Writing Tool, Capability Models, Standards Cross Index, Component/System Reliability Data, Surveillance Writing Tools System Life-Cycle Deploy System-wide life-cycle analysis and reasoning to identify and eliminate risks Monitoring Risk Increasing the level of surveillance as necessary Awareness, Supplier Assessment System, Improved Delegation Process, Outreach to University/Non-Profits, Conferences, Industry Coordination, Management Meetings, Risk Based Surveillance Training

5. Project Team Contract/Grant Acquisition Team Performance Solicitation Rev Contract Award What is the Risk Based Quality Process? Ongoing Technical/Program Risk Assessments 1 Risk of Requirements Contract/Grant Evolution Solicitation Development Surveillance 2 Risk of Performance Cost, Tech, Schedule “Level” of Surveillance Based on Risk

6. How Does Risk Based Quality Evolve During the Product/Process Life Cycle? Formulation Implementation SRR PDR CDR Production FRR Operations Production/ Acceptance/ Manufacturing COFR End Package Item Top- System Specs. Level Spec. Rqmts . Close Out S WBS WBS WBS Solicitation Trades Analysis H/W & S/W Detailed Concepts Verification Prelim. Drawings Drawings Plan Evolving Risk Identified Risk Profile Identified Risk

7. Organizational Evaluation Technical Risk Fault Tree - FMEA PRA Other Models Schedule Risk Cost Risk How Do We Evaluate “Risk of Requirements” ? Acquisition Team - Project Team Contract Risk Profile Accepted Risk Requirements Risk Evaluation Start with Solicitation Development Process Evaluation System Evaluation Data Heritage Procurement Finance Risk Lessons Learned Component Failure History Tech Reports PRACA TRL Evaluation

8. Technical Risk Assessments Reporting Analysis Schedule Risk Cost Risk How Do We Evaluate Risk of Performance”? Acquisition Team - Project Team Contractor Risk Profile Accepted Risk Performance Risk Evaluation Start with Contract Award Contractor Capability Contractor Performance Supplier Performance Evaluation Data Property Data Audits DCMA DCAA GIDEP Alerts Project Reports Reviews Financial Reporting Procurement Data

9. Safety and Health R & M Quality Where Does Risk Based Quality Fit Within NASA Policy? NASA Strategic Plan NPD 1000.1 NASA Strategic Management Handbook NPG 1000.2 NASA Organization NPG 1000.3 SMA for Spacecraft, Instrument, ELVs NASA Policy for Safety and Mission Success NPD 8700.1 http://www.hq.nasa.gov/office/codeq/doctree/qdoc.htm

10. Why Do Risk Based Quality Initiatives Focus on Acquisition/Awards? Cost of Contracts, Grants and Cooperative Agreements (Includes $1.3B for JPL) All other NASA cost such as payroll and facilities 19% 81% Over 90% of NASA’s FY 2000 Contract Obligations Utilized Performance Based Contracting $13.6 Billion Total Appropriation NASA FY 2000 Appropriations vs. Procurement Awards

11. Why is Industry Involvement Important? US Aerospace Industry Sales by Product Group (2000) Mil Air 23% NASA is about 40% of U.S. space sales, and 9% of the overall U.S. Aerospace spending Other 16% Aviation is 55% percent of U.S. Aerospace Spending Space 22% Civil Air 32% Missile 7% Aerospace Spending is $144 Billion

12. What Are the Challenges for Contracts? NASA Major Primes 60% to 80% of activity is sub-contracted Tier 1Vehicle / Airframe /Propulsion Manufacture Requirements Tier 2 Eight Thousand Active Contracts Integrators / Source Control/ Software Dev./ Products and Services Major Assemblies Tier 3 - Integrators / Source Control / Specialty Electronics / Wiring / Compnts Tier 4 - Make to Print / Machine Shops Shared Supply Base / Processors Tier 5 - Distributors (Fasteners, Raw Materials, Commodities, Adhesives, Special Materials) Tier 6 - Raw Materials (Castings, Forgings, Sheet, etc.) AEROSPACE SUPPLY CHAIN

13. What Are the Challenges for Non-Profits and Educational Institutes? FY 2000 Active Awards Estimated Value at Non-Profit and Educational Institutes 66 University Awards above $5million 7,375 Grants $2.0 Billion Total 1,414 Cooperative Agreements $2.9 billion total 400,000,000 350,000,000 300,000,000 250,000,000 200,000,000 Active R&D Awards 150,000,000 100,000,000 50,000,000 0 UNIV MIAMI UNIV FLORIDA BOSTON UNIV UNIV ARIZONA COLUMBIA UNIV UNIV CALIF IRVINE UNIV TEXAS AUSTIN OREGON STATE UNIV UNIV NEW HAMPSHIRE UNIV CALIF BERKELEY SUMMA TECHNOLOGY INC UNIV WISCONSIN MADISON UNIV COLORADO BOULDER UNIV COLORADO BOULDER RESEARCH TRIANGLE INSTITUTE MASS INSTITUTE TECHNOLOGY UNIV ALABAMA HUNTSVILLE UNIVERSITIES SPACE RESEARCH UNIV CALIF SANTA BARBARA SOUTHWEST RESEARCH INSTITUTE GEORGIA TECH APPLIED RES CORP UNIV MARYLAND COLLEGE PARK FY 2000

14. Knowledge PBMA Filming Technical Questions NEPAG EEE Parts Risk Management Vendor Status Problem Advisory Supplier Assessment System Capability Models Standards Index Check sheets Maintain Risk Based Surveillance Surveillance web site Surveillance training Quality Planning Policy Training Interviewing Tool Writing Tool Workmanship Pictorial Reference Application Notes Testing What Are Some of the Risk BasedQuality Task In Work? Coordination Quality Leadership Forum Industry / Associations (AIA, ADQMC, AAQG) EMC, PMC, S&MA, Procurement DCMA ISO TC 20 SC 14 NASA/ESA/NASDA Outreach Process Control Awareness Quality “Road Show”

15. What are the QLF Expectations? • Today we will be discussing many of these risk based quality tasks. • Many have just started. • There are many more at the Center level that have not been mentioned. • The Quality Leadership Forum will continue to play an active role in guiding the development of these activities and sharing ideas.