Learning Lessons at NASA -Goddard

1. NASA Goddard Space Flight Center Learning Lessons at NASA -Goddard January 22, 2008 “It is difficult to say what is impossible, for the dream of yesterday is the hope of today and the reality of tomorrow.” Dr. Robert H. Goddard1882 - 1945

2. Why Do We Need to Manage Knowledge? http://mediastream.ndc.nasa.gov/Public/KM/case_studies/failure_to_listen/default.html?iframe=true&width=100%&height=100%

3. What Have We Done?How Did We Do It?What Have I Learned?May 2003 to October 2011 Edward W. Rogers October 19-20, 2011 GSFC Chief Knowledge Officer Office of the Director Goddard Space Flight Center “Your Knowledge Management Programs should be like a good pair of shoes—they should fit your organization well and they should take you someplace interesting.” Ed Rogers, 2009

4. What Some Say… “If I hadn’t had to prepare for that workshop, I wouldn’t really have figured out what I learned from that mission.” Project Scientist at GSFC “That one PaL probably saved my project. “ Project Manager at GSFC “When one is buried deep in a project, one loses sight of everything else—other opportunities, visions, culture, GSFC as a whole. The course has given me a vast overview of GSFC that will allow me to be a more efficient employee. There are so many elements behind Goddard’s history that I really didn’t know. Knowing the history is even more helpful and exciting as a member of the NASA team.” RTMS Participants 2011 “Most effective part was the first two case studies. I really enjoyed hearing about other contracts & what they have encountered & how they’ve worked through them.” Procurement Forum, 2011

5. What Have We Done in Numbers? • Developed an Internal Case Study Collection • Over 50 Case Studies Developed and Published: Quarterly Case Catalog • Person Case Interactions (one person reading and learning from a case) • NASA and External Use of Goddard Developed Case Studies • 1000’s through APPEL NASA HQ Programs (total unknown) • 3000 at PMChallenge NASA and Contractor Public Conferences • Unknown Academic use outside of NASA through public release • Goddard Application of Lessons Through Cases • 5000 at Road to Mission Success at Goddard (Ten series taught) • 1000 at Various Goddard Open Knowledge Workshops • 600 with Flight Project Managers • 400 during Goddard Safety Weeks • 150 during Procurement Training • PaLs done with Projects, Teams and Org. Units • Advanced Concepts and Formulation Office captured over 150 key insights from PaLs done after proposal efforts • WIKI’s within Engineering Directorate for bench level sharing • 100’s of pages created and updated regularly

6. COLUMBIA Accident Investigation Board Costly Lessons About Learning The Challenger Launch Decision

7. The Problem is a Failure to Function as a Good Learning Organization “Shuttle management declined to have the crew inspect the Orbiter for damage, declined to request on-orbit imaging, and ultimately discounted the possibility of a burn-through.” “The Board views the failure to do so as an illustration of the lack of institutional memory in the Space Shuttle Program that supports the Board’s claim… that NASA is not functioning as a learning organization.” CAIB Report (2003) Section 6.1, Page 127 Reasons: Too busy, low value of typical KM activities, bureaucracy

8. Find the Leverage Point in the Equation Loss of Inflow of Customer Opportunity Human Human Need & Tools Capital Capital Focus ( ) - = Organization Rate of Rate of Potential Interaction Knowledge Knowledge Knowledge Coefficient Loss Discovery Utilization Openness Innovation & Knowledge Project & Sharing Creativity Decay Execution

9. Translating KM for NASA • Reliable Project Processes • Team make up (members) should be independent of quality of knowledge applied to the project • Organizational communication processes must reduce risk for the project • Knowledge flow loops must be shorter than project life cycles to facilitate timely learning • Sustainable Competency Base • Replenishing of social networks to support knowledge sharing as knowledge expertise ages • Knowledge sharing legacy systems adapted to modern work structures, protocols and practices • Effective mentoring and knowledge sharing to attract and retain the future competency base for the Agency

10. Address KM as a Learning Challenge • Lessons are learned when we reflect on our experiences • If we don’t stop and reflect: we generally don’t learn much • Reflection helps us overcome near-miss-bias: see HBR, April 2011 • Helps us learn from what we did right • The best time to reflect is right after the experience • Rules are a reflection of what we think will work • Often derived from failures • Tested and flown is the standard “Civilization advances by extending the number of important operations which we can perform without thinking about them.” Introduction to Mathematics Alfred North Whitehead, 1911

11. Must Understand System Tensions

12. Must Understand Our Success Model

13. Must Augment How We Do Learn

14. G O D D A R D S P A C E F L I G H T C E N T E R Simply, Start Smart, Stay Relevant Early Learning Zone Formal LL Zone

15. Draw a Picture and Talk It

16. Write a Learning Plan and Revise It • Goals of Learning Plan • Manage Knowledge Assets Efficiently • Facilitate Effective Knowledge Use • Build a Learning Organization Culture • Learning Practices • Pause and Learn • Sharing Workshops • Case Studies • Lessons Learned • Case-Based Training • Experience-Based Design Rules “The Goddard Plan is designed to overcome the previous Agency focus on IT as a KM driver with its over-emphasis on capturingknowledgefromworkers for the organization and instead focuses on facilitating knowledgesharingamongworkers.”p5 of draft Goddard Learning Plan

17. Pause And Learn Sessions (PAL) • Pause and Learn is a team based reflection activity for the team/group to process their own learning. • Five key parameters: • Non-attribution environment • Not a reporting activity • Close to the action (work level) • Participant discussion • Objective facilitator • Inserted into process • After major reviews/gateways • After incidents or mishaps • When team seems at risk • Participants have all expressed the benefit of the process to them and their teams.

19. Case Studies • We write good reports when things go wrong but, we don’t write much when things go right and we don’t write anything when we were just lucky. • Case Studies are a channel for learning • Collects ambiguous wisdom into a context • Opportunity for protagonists to share their stories • Proven methodology for hundreds of years • We need to be better at examining why things went right and not just assuming it was always because of our great planning and execution. • Cases help overcome a reticence to discuss things that were less than perfect but still worked which are a great learning opportunities. • “In fact, most people learn from others, learn by doing and learn from stories. … Most training is almost worthless.” • Interview with E-Learn, Larry Prusak, 2008

20. Case Study Products Write Them Teach Them Publish Them Wrap Them

21. Case Study Best Practice The Case Study Participants Facilitator Protagonist Teaching Note Exercise Handout Video Clips

22. Knowledge Sharing Workshops • Usually a project team (instrument, engineering, mgmt. science etc.) • Share in a panel discussion format their personal experiences • Keep it focused on “What I learned” • Sometimes use a case study or draft case study to spark discussion • Facilitated by a Moderator (usually the CKO) • Lasts about 2 hours • Sometimes needed after an incident/mishap etc. to get the real story • Help bring out personal lessons “If I hadn’t had to prepare for that workshop, I wouldn’t really have figured out what I learned from that mission. We must be learning from all our missions. “ Project Scientist at GSFC

23. The Road to Mission Success Workshop Series Objectives • Engage in dialogue with senior management on “Mission Success: The Way Goddard Does Business” • Articulate a clear and consistent statement of the rules, processes, and values that contribute to Goddard’s success with flight missions • Identify the array of support mechanisms in place at Goddard • Expand and empower the cadre of existing and potential leaders and managers within Goddard

24. Goddard Design Rules A search on the term “FPGA” will yield: FPGA GOLD Rule 2.10 – Electronic Design Link to specific NASA LLIS data Link to specific MIL-STD Link to klabs.org page on FPGA design Link to JPL Blue Book FPGA rules Link to relevant case study

25. What Have I Learned? “It’s not what you don’t know that will kill you. It’s what you think you know that just isn’t so.” Mark Twain

26. Learn from Decisions: Not Just Outcomes Success No Lessons Learned No Lessons Learned Mishap Investigation with Technical Lessons Learned Wrong Lessons Learned Failure

27. Good Organizational Depth Perception If people have a good grasp of what the organization is all about they will be more likely to openly share and communicate with each other. I can see how what I know matters to others. Leader led training !

28. Open Communications If people are satisfied with the communication systems and processes in place they will be more likely to openly share and communicate with each other. If I speak something happens. Facilitated discussions !

29. Reward Fairly (and Punish Fairly) If people perceive the organizational employment game to be fair and open they will be more likely to openly share and communicate with each other. If I work hard, it matters. Career plans and IDPs !

30. Most People Love to LearnMany Fewer Want to Be Taught, Told or ControlledInspire People with Knowledge and You Will Live Long and Prosper January 22, 2008