The Future Role of Quality in Aerospace: Improving the Chances for Mission Success

1. The Future Role of Quality in Aerospace: Improving the Chances for Mission Success International Aerospace Quality Group 3rd IAQG General Assembly Meeting Kyoto, Japan March 14, 2002 Dr. Michael A. Greenfield Acting Associate Administrator Office of Safety and Mission Assurance (1)

2. The Big Picture – What NASA Must Do • NASA’s programs must contribute to national priorities: • Economic growth and security. • Increased understanding of science and technology. • Protection of Earth’s environment. • Educational excellence. • Peaceful exploration and discovery. • NASA’s programs must continue to build on the Agency’s core foundation for aeronautics and aerospace research and development. • NASA must demonstrate improvement in business practices.

3. Stakeholder Expectations Programs that are on time, within budget, and are safe/successful. High priority planetary exploration, climate research, space station experimentation, and biological science that is effective. New technology for improved access to space, and more capable planetary missions. Outcomes that are meaningful to the public. Business Realities Flat budgets. Workforce retirement, loss of critical skills. Aging infrastructure. Cost growth. Increased reliance on the performance of the supply base. Our Current Environment

4. To succeed, processes must be stable, capable, and in control.

5. Defenses Are Never Perfect We perceive our ideal system of defenses like this. Hazard But the reality can look like this. Source: James Reason, Managing the Risks of Organizational Accidents, 1997, p. 9

6. When Events Lines Up Just Right, the Consequences Can Be Devastating Requirements Hazard Design Process Development Manufacturing Test Operations Mishap Adapted from : James Reason, Managing the Risks of Organizational Accidents, 1997, p. 12

7. Success Depends on Proactive Quality Efforts • We need to understand that our current defenses are not invincible. • We need to proactively build in ways to “close the holes.” • Quality-based tools are the best bet. • Requirements Definition • Process Control • Surveillance • Independent Analysis • Peer Review • Other Requirements Design Quality-based tools have high leverage. Product inspections have limited impact. They address the “symptoms.” Adapted from : James Reason, Managing the Risks of Organizational Accidents, 1997, p. 12

8. Improve Acquisition • Standards/Clauses • (AS 9100) • Tailored Quality Requirements Maintain NASA’s Quality Approach Improve Performance Improve Oversight + + • Performance Based • Risk Based • Industry Based • 2nd, 3rd, and other • Capability Maturity Models • Process Control • Continual Improvement Address High Risk Processes Address High Risk Suppliers Address High Risk Parts + + • EEE Parts Risk Mgmt. • Vendor Status • Problem Advisory • Awareness • Application Information • Testing • Risk based supplier assessment by technology area Industry Partnership (Quality Leadership Forum, Industry Associations) Outreach to Supply Base (Awareness, Tools, Best Practices, Forums, Conferences)

9. Integrated Risk Approach Quality/Procurement/Program Management Acquisition Planning Solicitation and Award Performance • Work Analysis • Develop WBS • Analyze risk of work • Performance Analysis • Select indicators, standards • Determine acceptable quality level • Analyze cost/quality tradeoffs • Risk Assessment • Use historical data, experts, brainstorming, checklists • Identify, analyze, quantify: risks, triggers, probability, impact, task linkage, surveillance method • Identify stakeholders • Develop Draft SOW • Surveillance • Monitor contractor performance • Conduct risk based assessments • Adjust surveillance and performance metrics based on changes in performance and risk • Communication • Manage communications per Surveillance, Communication and Risk Management • Project Risk Management • Monitor • Track • Adjust • Solicitation • Release Draft SOW • Analyze industry comments • Develop acquisition strategy • Issue final RFP • Proposal Evaluation • Assess risk for each proposal • Contract Negotiation • Surveillance method and frequency • Performance metrics • Program management plan • Risk allocation • Risk Management • Includes Surveillance/ Communication • Contract Award

10. Consistent Requirements Tailored to Risk Today A proliferation of different program/contract specific words used to express the same requirement For example “Limited Life” Will Become Single set of recommended words Guidance to assure the requirement is applied in risk areas

11. Trades Aren’t Simple: Risk Versus Total Cost of COTS

12. To Improve Quality, Management Must Also Improve • Establish Accountability • Industry based requirements • Performance based contracts • Industry self oversight • Emphasize Basics • Addressing systemic issues • Demand Evidence • Oversight based on risk • Impose Consequences for Poor Performance • Award fee, incentive fee, future business • Focus on Results • Mission success criteria Mission Checklist • Clear Expectations • Clear Accountability • Clear Results

13. What’s the Bottom Line? • There’s a lot at stake: • Human life. • One of a kind hardware. • Scientific knowledge. • International cooperation in space. • Economic viability. • NASA must focus on its core competencies – R&D. • Quality is critical to our ability to meet our ambitious goals.

14. The Future State of Quality The future state of quality will have these elements: • Quality requirements and expectations that are proportional to the risk of the activity and its expected outcome. • A supplier oversight strategy that is integral to comprehensive program management (cost, schedule, and technical). • Emphasis on high risk areas such as EEE parts, printed wiring board production, and other similar space technology/ environment concerns. With clear requirements, focused oversight, and proper emphasis on risk, we increase the chances for mission success, and everybody wins.