Surviving and Thriving on NASA Audits…. One Organization’s Story of Turn Around and Success

1. Surviving and Thriving on NASA Audits….One Organization’s StoryofTurn Around and Success Emily Bird Broderick Safety and Quality Assurance Space Department emily.broderick@jhuapl.edu

2. Divisions ofThe Johns Hopkins University School of Arts & Sciences Whiting School of Engineering School of Professional Studies in Business & Education School of Hygiene & Public Health School of Medicine School of Nursing Applied Physics Laboratory Nitze School of Advanced International Studies Peabody Institute

3. The Johns Hopkins University Applied Physics Laboratory • The Applied Physics Laboratory (APL) is a not-for-profit, federally chartered University Affiliated Research Center (UARC) for engineering, research and development. • Our 399-acre campus is home to more than 4,200 women and men working on more than 400 programs that protect our homeland and advance the nation's vision in research and space science. • The core purpose of JHU/APL is to enhance the security of the nation through the application of science & technology. • APL is firmly committed to space science & technology and other challenging areas that are also synergistic with our core purpose. The Lab also carries on important work in areas (e.g., biomedicine) that are synergistic with JHU as a whole. • Our goal is to make critical contributions to critical challenges.

4. The Applied Physics LaboratorySpace Department • The APL Space Department has successfully developed and flown over 150 space science instruments since 1970. • We have assisted NASA in space exploration from the Sun to Pluto, most recently with the STEREO and New Horizons missions. • Currently, the Space Department has over 35 active projects and programs ranging from National Security Space studies of operational conceptions to full NASA space missions such as Radiation Belt Storm Probes (RBSP).

5. The APL Space DepartmentSafety & Quality Assurance Group (SQA) SQA is responsible for Mission Assurance activities including: • Systems Assurance Management • Hardware Quality Engineering • Software Assurance • System Safety Engineering • Configuration Management • Quality Inspection • Supplier Quality Management • Internal Audits • ESD Compliance and Training • Operator and Inspector Workmanship Certification • AS9100/ISO Implementation • Space Department Staff Training and Certification Records • Corrective Action Management and Tracking • Metrics Development and Analysis • Process Improvement Initiatives

6. Setting the Stage… • Assessments were conducted by NASA’s team at APL in 2003 and 2005. Over 75 findings were issued (not counting observations) during the two assessments. • APL received “red” and “yellow” scores with report findings such as: “Top management commitment to quality and safety is not apparent.” “Development and maintenance of formal quality records, plans, and procedures are incomplete and inconsistent.” “No formal corrective/preventative action process is in evidence.” “The demonstrated level of commitment to a systems approach to Safety/Mission Assurance falls short of supporting NASA’s commitment of Safety First.”

7. Facing the Challenge… • In June of 2007, APL still had over 60 open items with NASA from the audits held 3-5 years prior. • While many topics had been addressed, there had been little follow-through with the sponsor, and little organization of responses to close the items. • The new team hired into APL was tasked with resolving the open issues and ensuring that APL’s Mission Assurance activities were sufficiently implemented and understood to ensure success upon NASA’s return assessment in 2008. • The mission objective from management was clear: “APL’s Safety and Mission Assurance activities must be as robust, internalized, standardized, and exceptional as our design, engineering, and science.”

8. Changing the Paradigm… • Management has to be committed to supporting Mission Assurance improvement initiatives. • Management includes Program Management, Engineering, Fabrication, Integration and Test, Facilities, Operations, Business Services, Human Resources, and IT. • In most places, Mission Assurance would not exist without everyone else. “They” make sure “we” have jobs. Treat them as your customers. • Your harshest critics are often those who have NEVER tried to play by the rules, and those who have ALWAYS try to play by the rules. Listen to both. • The most critical managers to reach and the most important staff to partner with are the ones in the trenches trying to get the work done. Partner with them whenever possible. • Be willing to do whatever it takes to make Mission Assurance successful. Be known as the team willing to go the extra mile. Roll up your sleeves and help.

9. Preparing for success at your location… • Start from where you are. • Clean up the past. • Set management expectations appropriately. • Approach the audit as a project and run it accordingly. • Identify your greatest risk areas and place more emphasis on fixing them. • Define a team of subject matter experts who will be critical during the audit. • Partner with the folks you need information from to help them prepare. • Meet regularly with your core implementation team to get status updates and give deadlines.

10. Preparing for success at your location… • Take the audit as an opportunity to get facilities involved with extra cleaning, pick-up, facility fixes, etc. • Throw stuff out. • Walk your facilities with your Safety Officers and Managers. • Hold All-Hands meetings to explain what NCAS is and what they will be doing. • Hold special sessions to educate staff on how the audit will be conducted based on how involved they will be. • Make certain no major system upgrades are planned close to the time of your audit. • Hold your “No’s”. • Encourage, reward, cheerlead, and smile.

11. Preparing for success with NCAS… • Believe the NCAS team when they say they are coming to help and partner. • Engage the NCAS team as early in the process as possible. • Ask NCAS questions about what tools will help you prepare for the audit. • Ask for NCAS assistance in areas where you need expertise. • Communicate openly with NCAS about what programs or activities will be in-scope for the audit. • Agree on the standards that will be used to conduct the audit (AS9100, ISO). • Ask for special visits to close out old findings.

12. Operating during the Audit… • Set up a “Mission Control” area for the audit team to congregate in. • Have your IT Department preload laptops with access to the tools needed to retrieve QMS documents and records. • Have copies of all contracts, SOWs, and related documents in the room. • Schedule your staff in blocks of time and ask them to come to you for the audit. • Create and distribute a contact list with staff cell phone numbers. • Prepare management to be available at the end of each day for the debrief. • Have facility owners meet you to walk the auditors through each area.

13. Taking steps after the audit… • Before the auditors leave, schedule a date for them to return and review your progress. • Enter findings in your Corrective Action system. • Meet with your management team to review all findings and assign proper personnel and levels of severity. • Track your progress toward completion weekly with notification of status to senior management. • Contact NCAS is you are unclear about an item after they have left rather than guessing about what they meant. • Don’t confuse corrective and preventive actions. Fix the problem found, then create a separate preventive action if there are other steps to take.

14. Celebrating success… • During the 2008 NCAS Assessment, APL received only 8 non-compliances, 21 observations, and 7 positive observations, all but four of which have already been implemented and found effective. • Report findings included comments such as: • During this audit, APL demonstrated that it is a model for organizations required to redefine and transform their safety, quality, and mission assurance systems.... • It is clear...that APL has fully embraced the need for a robust quality management system and has put the necessary staff and resources in place to support their commitment to continuous improvement. • ...more positive observations were cited during this APL audit than during any supplier audit in recent memory. These systemic improvements, are a sign of the laboratory’s commitment to excellence.