NPR 8621.1C Status

1. NPR 8621.1C Status NASA Safety & Health Managers’ Meeting March 27, 2013 Steve Lilley NASA Safety Center

3. Your handout is a preview to the draft When 8621.1C is published, it will be more extensive

4. NPR 8621.1 Revision Process History • August 2011 – Kickoff and Step 1: Justify requirements as a set - Charter the committee - Cost/ benefit analysis • September 2011 – March 2012: Step 2: scrub all 461 requirements X 9 separate tests -Restack Chapter 1 requirements -Necessary? Achievable? Verifiable? • March 2012 -- July 2012: Step 3: Outline and First Draft - “Gang of Five” detailed scrub and streamlining - Revisit teams for V & V • July 2012 -- August 2012 – Technical Review by Mishap Program Working Group Goals: find and fix showstoppers. Verify the streamlined organization is more usable. • August-November 2012 –Step 4: “Gang of Five” statuses ALL 836 MPWG comments • Traceable on NSCKN - Project schedule drove G of 5 to finish by Nov 30 • November 2012-March 2013: Step 4 at OSMA: independent V&V of second draft • Second Directives Control Board • April 2013-Draft into NODIS • OSMA tech writer formatting for NODIS entry

5. Thanks to the team • Isi Greenfeld • Don Hall • Norma Horton • John LaPointe • Lynne Lee • Dave Loyd • Jimmy McLaughlin • Jon Mullin • Rick Parker • Debbie Ripley • Rebecca Selvage • Dan Thomas • Casey Tull Marv Bunting Marilyn Chambers Dan Clem Chris Counts Izzy Davila Beth Dickey Manny Dominguez Kristen Easton Lou Ann Fikes Jennifer Franzo Wayne Frazier Kristie French Laurel Gaab Mark George

6. New graphics in the draft We split the Figure 2R&R table, in two: New New

7. New graphics in the draft Old New Replaced table and lots of text with one self- explanatory table

8. New graphics in the draft-Mishap Report Outbrief Process-1

9. New graphics in the draft- Mishap Report Outbrief Process-2

10. New graphics in the draft-current IRIS reporting requirements Current guidance on everything required for IRIS entry. Several data elements are called out, but overall guidance is open to much interpretation.

11. New graphics in the draft-Incident reporting 1 Ensures 29 CFR 1904 and NPR 8621.1C investigation requirements are met.Ensures that, regardless of the system used, an agency-wide data set is published and traceable. Provides much-improved ability to understand causal scenarios behind high-risk incidents of any classification. Supports modern safety management systems in use today by NASA (VPP, ANSI Z-10, ICAO SMS)

12. New graphics in the draft-Incident reporting 2 All elements were agreed upon at the MPWG Technical Review in August 2012. Who-What-When-Where-Why justification. This concludes the 8621.1C update. Thanks for your time! Are there any questions?

13. Backup

14. Requirements, Coming and Going • Revision process began August 11, 2011 at MIWG annual meeting. • By July 6, 2012, the following had been accomplished: • Teams of investigative experts and senior staff had reviewed all 461 requirements and surrounding policy language in detail to establish necessity, attainability and verifiability. • 115 requirements were identified for removal, most as redundant. • 29 new requirements were identified for inclusion based on inputs from each Center. • All remaining requirements were re-stacked in functional order. Roles and responsibilities were collected and called out for each step in the process. • 193 requirements-59 redundant = 134 requirements in the current Chapter 1 overview were flowed into each process step and Chapter 1 was eliminated.

15. Chapter-by-Chapter Highlights • Current Chapter 1 Approximate Requirements flow:

16. Readiness to Conduct Investigations • Current Chapter 2 will be the new Chapter 1. • Highlights: • Mishap Preparedness Contingency Plan section becomes one list of plan topics, with Center vs. Program guidance broken out. • Readiness Roles and Responsibilities all together in one section. • All training is consolidated into this chapter, recognizing differences between knowledge and skills needed in the field versus administratively. • 34 redundant requirements removed, 2 new requirements added.

17. Initial Response to a Mishap or Close Call • Current Chapter 3 will be the new Chapter 2. • Highlights: • NASA mishap classification criteria changed to match DoD. • NTSB notification scenarios for aircraft increased from 11 to 16. • IRT Roles and Responsibilities: • IRT Lead R&R added • Contractors may collect non-privileged witness statements • New guidance for Human Test Subject incidents. • Coordination with OCHMO and Institutional Review Board process per 45 CFR 46.103 • 30 redundant requirements removed.

18. Select the Investigating Authority and Support • Current Chapter 4 will be the new Chapter 3. • Highlights: • Simplified table to identify the cognizant Appointing Official per mishap classification and involved personnel/property. • Re-organized and clarified selection criteria for Investigating Authority and members. • 14 redundant requirements removed, 9 new requirements added.

19. Mishap Investigation Process • Current Chapter 5 will be the new Chapter 4. • Highlights: • Evidence Collection: instituted defined concept of ‘evidence’ instead of ‘data’ • Simplified or removed much non-requirement advice regarding evidence collection (now implemented in formal training) • Determine Findings: added analytic rigor to outcome requirements, yet maintained flexibility on analytic methods. • MWAR procedures added. • 30-day Report requirements specified. • Recommendations: should assess and rate for mishap-preventive effectiveness to assist AO post-mishap. • 14 redundant requirements removed, 9 new requirements added.

20. Mishap Report • Current Chapter 6 will be the new Chapter 5. • Highlights: • Streamlined A/B/HV endorsement process at the Outbrief. • Public Release Review procedures codified. • Mishap Summary format and review process established. • IRIS Common Data Elements identified. • 18 redundant requirements removed, 7 new requirements added.

21. Post-Investigation Activities • Current Chapter 7 will be the new Chapter 6. • Highlights: • Rationale for requirements better-documented. • 5 redundant requirements removed, 2 new requirements added.

22. Definitions 17 new definitions: • Control (per NASA RCA training) • Evidence (physical, demonstrable, testimonial, documentary) • Hazard (per NASA-STD for SMA terms) • Human Error (per NASA RCA training) • Initiating Event (per NASA RCA training) • Intent for flight (per DoD) • Launch (per NASA-STD for SMA terms) • NASA Aircraft (per NPR 7900.3) • Probable cause (per Center input) • Procedure (per NASA-STD for SMA terms) • Process (per NASA-STD for SMA terms) • Range (per NASA-STD for SMA terms) • Risk (per NASA-STD for SMA terms) • Spacecraft (per NASA-STD for SMA terms) • Test (per NASA-STD for SMA terms) • Timeline (per NASA RCA training) • Undesired Outcome (per NASA RCA training)

23. Definitions continued 4 Re-worded definitions: • Aircraft mishap >>Aircraft flight or ground mishap • Cause (includes precedence, necessity, relational tests) • Executive Summary (expanded) • Human Factors (expanded per NASA RCA training)

24. Methodology & Assumptions In support of the review and revision of NPR 8621.1, this cost – benefit analysis is developed as part of the “Requirements Review & Evaluation Process” to characterize overall effort, necessity, and effectiveness of the NASA Mishap Investigation process. Each mishap investigation requires dedicated performance from defined resources over a defined period of time. An additional 50% margin is added to this estimate to account for consultation, endorsement and critique. Investigation hours are averaged across a 5-year history. Estimates of mishap consequences are based on National Safety Council (NSC) estimates of disabling injuries (including permanent disability, partial disability, or lost time) costing approximately $50,000. Property damage costs are directly recorded. A 2008 RAND study estimates that a comprehensive safety and health standard results in 20-40% reduction in workplace injuries beyond common compliance. A similar study by OSHA, and NASA’s agency-wide incident rate both confirm 50%-60% mishap avoidance below comparable industry. It is assumed that the mishap investigation process is responsible for a portion of the mishap prevention (or subsequent mishap recurrence) benefit derived from the RAND, OSHA and NASA estimates.

25. Investigation Costs

26. Consequence Costs

27. Benefits Based on the total cost of mishaps consequences during the 2006 – 2010 period ($414 million), NASA loses approximately $83 million each year to mishaps. Based on RAND study, it is estimated that comprehensive safety and health program activities can be credited with preventing 20-40% of mishaps beyond regulatory compliance. NASA’s performance is over 60% lower than comparable industry mishap rates. A 50% mishap prevention improvement is used for this analysis. As much as $83 million in additional mishap costs could be incurred by the agency each year (where $166 million represents theoretical un-mitigated mishap cost).

28. Conclusions Based on this analysis, the mishap investigation process costs NASA approximately $5 million annually in direct investigation resources. The mishaps themselves have averaged an estimated $83 million annually in debilitating injuries and property damage. Together this represents $88 million annually in direct mishap costs. There are significant additional costs in administration, corrective action, and mission impact. Based on RAND and OSHA estimates and NASA performance, the highest performing safety and health programs realize as much as 40-60% improvement in mishap prevention performance over those organizations choosing to rely on regulatory compliance alone. With this in mind, it is estimated that a sound investigation process can contribute approximately 22% to the full cost avoidance estimate, or approximately $18.2 million annually. The close call proportion of the mishap investigation process represents a very small relative investigation investment – 7% of investigation costs – to learn from the largest overall proportion of opportunities – 61% of all mishap events – in the 2006 to 2010 period. Theoretically, the close call proportion of the investigation process could be responsible for as much as $11.1 million of the estimated annual cost avoidance.