Bob Stuewe Operating Experience Coordinator Contractor Assurance Office – Performance Feedback

1. Learning About LearningProgress in Lessons Learned at Los Alamos National Laboratory presented to theDOE Operating Experience Coordinators WorkshopMay 5, 2010 Bob Stuewe Operating Experience Coordinator Contractor Assurance Office – Performance Feedback Los Alamos National Laboratory

2. Bottom LineResults Take About Three Years to Manifest End of CY 2006 FY 2010 (Now) 800 processed per year Pass preliminary screening Multiple pre-planned sources All program/function areas 150-200 internal per year 1500-3000 web users/month >2100 in searchable archive >30,000 document accesses per month 100-225 searches/month Analyze/Trend Occurrences, Causes Track Actions (risk-based) 2.5 FTEs 1.2 FTE on Analysis • 163 processed per year • Pass preliminary screening • Nearly all DOE/HSS • Safety • 10 internal per year • 200 web users/month • 250 in searchable archive • ?? document accesses per month • ?? searches/month • Trend Occurrences • Track Communications • 2 FTEs • 1 FTE on the Trending

3. Learning About Learning Lessons About Effective Lessons Learned Programs • Champion Learning, Not Lessons • Reliable Processes Build Trust and Use • Clarify Roles to Eliminate Redundancy • Support Work Processes • Enable Risk Management • More Focused Equals More Effective • Build and Use Indicators of Effectiveness • Encourage Innovation

4. Champion Learning Not LessonsOperating Experience Programs are Key to Knowledge Management World of Possible Knowledge Operating Experience Program Learning happens in the work unit in many, many ways (Required reading, training, requirements updates, work team meetings, plan of the day, procedure updates, learning teams, process improvement teams, corrective action teams etc.)

5. Champion Learning Not LessonsFrom the Criteria for Performance Excellence (MBNQA) • Learning is one of the three pillars of organizational performance. • Learning is referenced 103 times in 88 pages • “The term “performance excellence” refers to an integrated approach to organizational performance management that results in (1) delivery of ever-improving value to customers and stakeholders, contributing to organizational sustainability; (2) improvement of overall organizational effectiveness and capabilities; and (3) organizational and personal learning.” • “To be effective, learning should be embedded in the way an organization operates.”

6. Champion Learning Not LessonsKnow How Your Organization Learns and Support the Methods • Nuclear and High Hazard Facilities • Conduct of Operations: required reading, procedures, FLM weekly meeting, critiques, facility lessons learned coordinators, etc. • Subject Areas • Electrical Safety Committee, Chemical Safety Committee, Glovebox Safety Committee, Security Incident Analysis, etc. • Lower Hazard Organizations • Worker Safety and Security Teams, Learning Teams, Management Meetings, mentoring, on-boarding, etc. • Institutional • Management review boards, requirements updates, training updates, institutional web pages, etc.

7. Reliable Processes Build Trust and UseTrust Requires Relationships – With Products/Services Used • Operating Experience and Lessons Learned Process • Collects, screens, risk-prioritizes, and distributes operating experiences and lessons learned from key external and internal sources using timely communications. • Three risk-priority levels with different disposition/communication pathways • Lessons Learned and Operating Experience Archive • Captures relevant lessons and best practices reported by the Laboratory managers and workers, contractors, subcontractors, and from other DOE sites and external sources. • Knowledge base • Mirror:Translating Events into Actionable Information • Communicates high-leverage actions, lessons learned and best practices associated with operating experience patterns and trends.

8. Reliable Processes Build Trust and UseProcess Management is Proven To Create Reliable Products/Services • Establish Process Management • Lean Six Sigma - Champion and Yellow Belt • Process Team (Weekly - Operations) • Integration Team (at least Annually - Integration and Strategy) • Benchmark (inside and outside DOE) • Define Product/Service Quality Specifications • Purpose -> Products/Services-> Customers-> Requirements -> Controlling characteristics • Defined processes (steps and roles) for each product (mapping) • Set up Process Controls and Improvement • Program Description, Procedures, Checklists, Tools • Use Contractor Assurance System: Goals, Measures, Assessments, Issues Mgt, Process Improvement • Establish Task Roles (daily, weekly, monthly, quarterly, annual) • Annual Improvement Plan (effectiveness and efficiency) • Execute and Innovate

9. Reliable Processes Build Trust and UseReliable Processes Requires Reliable Operations Thursday Screening Team Process Team and occasionally the site office Finalize screening Determine dispositions Write headlines Publish Weekly Summary Email Friday Week Close Out In case extra time is needed to complete the weeks deliverables Backlogs are deadly Monday Process Team Updates Upcoming Communications Plan Operations Improvement Plan Wednesday MIRROR Team Organized by sections of the publication Quality review and improve key messages and actions Design year-end and next issue Tuesday Integration CAS Management Team Institutional Management Review Board As Needed • Daily intake and preliminary screening • Work on products and services (MIRROR, lessons, support customers) • Work on Communications Plan and Improvements Plan

10. Clarify Roles to Eliminate RedundancyFunctional Approaches Increased Costs, Decreased Effectiveness • LL requirements embedded in many directives, guides • Alignment and Integration can save taxpayer money • LANL approach • OPEX program supports multiple functions – use an integration team to plan annual improvement (Safety, Security, Env Mgt, Project Mgt, Facilities, HPI, Training) • Issue & Corrective Action Management - handles extent of condition for institutional/local, OPEX program helps with what’s in between • QA/SCI – handles work-related suspect, counterfeit and defective items, OPEX serves as knowledge base, get directly involved at Urgent level • Planned integration with internal security LL and Security Lessons Learned Center • Organizational and Subject Area LL Knowledge Bases and Process align with and link to Institutional • Contractor Assurance System: Measures, Assessment Planning, Issue Effectiveness Evaluation • Site office uses LANL screening and disposition process, oversight of higher risk

11. Support the Work ProcessesEasy to Think About the “Push”

12. Support the Work ProcessesWork is Always Underway – Don’t Try to Change Cars While Moving

13. Support the Work ProcessesBeware of the Adverse Unintended Consequences of the “Push” • Common Lesson from OPEX: Not Thinking Through a Change • Nuclear Facilities • Procedures control the quality • Modifying procedures include many steps, for good reason • Conservation of Attention Law • Only so much time in a day • Rushing to implement a lessons learned can create rework • Resource Allocation is Not an OPEX Role • Acting on a lesson competes for resources • OPEX Coordinators do not know the daily priorities on the work floor

14. Support the Work ProcessesHard to Think about the “Pull” – But Equally Important

15. MS Sharepoint Reduced Cost IT Costs Process Costs Aligns with Work Processes Topical Libraries (key systems) Alerts (individual and topical groups) How-To Tools Full PDF search capability Support the Work ProcessesSupporting Work Processes Requires a Knowledge Base

16. Enable Risk Management To Engage Senior Management – Help Reduce Risk • Hazard inventory is pretty stable at the sites • Sharing known or low hazards provides little value to management • “New” and “Significant” Hazards important and valued • Risk Management • About reducing consequence or probability • Risk Management • What is the risk? (to the mission) • How large are the potential adverse consequences? • What are you doing to handle it? • How likely are you to succeed? • Should you do more? • Requires knowledge of organizational processes • Russian Hydro as an Example Where Lessons Learned can add the most value to management.

17. Enable Risk ManagementRisk-based Prioritization Matrix derived from O210.2 with Security from IMI levelsUrgent and Caution matrix essentially identical to Order – will add Best Practice un future Urgent=Immediateaction is needed to ensure that the Laboratory is in a safe, secure, and environmentally sound mode relative to the information identified in the alert. Active engagement of relevant manager(s). Formal issues and corrective action tracking. Work Planning Processes (Line) Training Development (Support) Requirements Development (Support) URGENT Monitor Sources Screen Clarify Communicate Caution Informational Caution = Timelyaction may be needed. Routine notification through Lessons Learned and Operating Experience Weekly Summary emails and LANL Home Page. LANL managers determine relevance and take needed action. Informational = Futureaction may be needed. Posted to LANL Home Page if broadly applicable and actions clear. Important to knowledge base.

18. Enable Risk Management Risk Levels Differ from Site to Site Urgent Lessons most commonly associated with DOE Safety Alerts and Advisories, defective safety items, and other sources

19. Enable Risk ManagementSelect Sources to Support Risk Management – Might Change over Time

20. More Focused Equals More Effective Tailor Communication Methods to Audience and Media Capability

21. More Focused Equals More Effective Weekly Summary Email – Management and LL Roles with Prioritization • Recipients • Division Leaders and above • NNSA Site Office • LL Roles and Upon Request • Augments daily ORPS • Specific = Relevant • Define Most Specific Customer • Select Words in Headline • Integrity Check • Order (First and Last Most Read) • Scan-able (6 most important/actionable; > 6 has lower read rate)

22. More Focused Equals More Effective LANL Homepage to Reach Workers Directly • Some Communication Overlap Helps Effectiveness • Main Users • Workers • Broadest lessons • Includes worker home safety • Specific = Relevant • Even shorter headlines • Pre-job brief is major customers

23. More Focused Equals More Effective Internal Lessons Learned Submission – Content over Format • Adobe Fill-able Form • Just the Facts • Word Docs too slow, costly, hard to share with DOE • Field Validation – reduce errors • Fields support processing • Screening and prioritizing • Interface to DOE/HSS • Includes picture objects • Submit Button linked to LL Archive and Process • Becoming standard at facility level

24. More Focused Equals More Effective LANL Mirror (Quarterly Publication) • Translating Events into Actionable Information • Events binned by systems and processes • Patterns and trends (incidents, occurrences, causes) • Year-End and Two-Year Running Reflection • Use statistics to identify changes/trends • High-leverage lessons/actions aligned with greatest performance gaps • Recognition for developing and sharing lessons learned

25. More Focused Equals More Effective LANL Mirror: Events Causes Checklist for Improvements

26. Build and Use Indicators of LL Effectiveness Fact-based Improvement is the Objective • Use CAS Dashboard • LL Indicators • Processing time performance • Urgent within 24 hours • Caution within 7 days • Document Access Rate • Search Rate • Knowledge Inventory • Internal LL Generation Rate • LL Program Execution • Routine Process Milestones • Progress on Improvement Plan • Corrective Actions

27. Build and Use Indicators of LL EffectivenessNeed measures than can help understand Cause and Effect

28. Build and Use Indicators of LL Effectiveness Need measures than can help measure use for Work Planning

29. Encourage InnovationLANL Performance Communication Center • Modernizing Communications • Chat with an Expert • Builds Knowledge Base • Blog • Role-based information • Relevant Tools • Relevant Requirements • Best Practices (coming soon) • Custom version of LL Form • Recognition

30. Encourage InnovationSub-Unit Approaches Evolve as their Learning Processes Mature • R&D key lessons included in “On-Boarding” materials for new hires (might become a best practice) • Local LL Archives Linked to Institutional Archive • Project Management • Engineering • D&D • Maintenance • Facilities • Learning Teams (linked to HPI implementation)

31. Learning About LearningProgress in Lessons Learned System at LANL Questions?