Root Cause Analysis and critical thought

1. Root Cause Analysis and critical thought Connecting the dots to deliver value-added results International Professional practices framework practice advisory 2320-2: Root Cause Analysis September 11, 2012

2. Speaker ProfileJames RoseCIA, CRMA, CPA, CISA, CISSP • Vice President & Chief Audit Executive at Humana, a publicly-traded health and wellness company headquartered in Louisville, Kentucky • Vice-Chair, International Professional Issues Committee of the Institute of Internal Auditors • Audit Committee Member, United Nations World Food Programme • Co-lead Data Analysis and Review Committee –Public/Private Healthcare Fraud Prevention Partnership with U.S. Department of Health and Human Services and U.S. Department of Justice • Humana’s Internal Audit Consulting Group consists of 75+ associates with diverse backgrounds in GRC systems, audit, consulting, technology, nursing, law, compliance, actuarial science, data governance, finance, project management, and investigations

3. Root Cause Analysis (RCA): A Brief History • Developed by Sakichi Toyoda who later became the founder of Toyota • First used during the development of Toyota’s manufacturing processes in 1958 • 5 Whys was the earliest method of RCA used • Motorola developed Six Sigma in 1986 using specific methods to outline a RCA

4. RCA In Relation To IIA Standards • Standard 2320: Analysis & Evaluation • Internal auditors must base conclusions and engagement results on appropriate analyses and evaluations • Practice Advisory 2320-1: Analytical Procedures • Practice Advisory 2320-2: Root Cause Analysis • Standard 2410: Criteria for Communicating • Communications must include the engagement’s objectives and scope as well as applicable conclusions, recommendations, and action plans • 2410-A1: “Final communication of engagement results must, where appropriate, contain the internal auditors’ opinion and/or conclusions. When issued, an opinion or conclusion must take account of the expectations of senior management, the board, and other stakeholders and must be supported by sufficient, reliable, relevant, and useful information.”

5. Critical Thinking and Insights Insight = Catalyst, Analyses, and Assessments Smith, T. and Miller, P. (2011). Research Results. In Insight: Delivering Value to Stakeholders (page 14). Altamonte Springs, FL: The Institute of Internal Auditors Research Foundation (IIARF).

6. Perceived Gap in Insight Delivery by Role Smith, T. and Miller, P. (2011). Research Results. In Insight: Delivering Value to Stakeholders (page 17). Altamonte Springs, FL: The Institute of Internal Auditors Research Foundation (IIARF).

7. Question 1To what degree is RCA applied in your audit engagements?

8. What is Root Cause Analysis? • Root cause analysis is defined as the identification of why an issue occurred (versus only identifying or reporting on the issue itself) • In this context, an issue is defined as a problem, error, instance of noncompliance, or missed opportunity • A core competency necessary for delivering insights is the ability to identify the need for root cause analysis and, as appropriate, actually facilitate, review, and/or conduct a root cause(s) analysis • Internal audit can be the ideal group to analyze issues and identify the root cause(s) given their independence, objectivity and cross-functional view • Root cause analysis benefits the organization by identifying the underlying cause(s) of an issue. The RCA provides the basis to resolve the true issue that – if left unmitigated – could impact the organization again in the future.

9. Illustrative Root Cause Analysis Techniques • “5 Whys” • Failure mode and effects analysis • SIPOC (suppliers, inputs, processes, outputs, customers) • Flowcharting of the process flow, system flow, and data flow • Fishbone diagrams • Critical to quality metrics • Pareto chart • Statistical correlation

10. 5 Whys • The practice of asking, five times, why the failure has occurred in order to get to the root cause/causes of the problem • Note: 5 is an arbitrary number, it may take more or less to get to the root cause of the issue that is reasonable. You should attempt to answer 5 why using multiple paths to ensure you have gotten to the root cause. 5 Whys Process • Write down the specific problem • Ask the first ‘Why’ and write the answer • Continue until what you believe is the true root cause is defined • Don’t allow an early plausible answer to keep you from continuing to ask why!

11. 5 Whys Example The City Veteran’s monument was disintegrating From the chemicals to clean pigeon poop They eat spiders and there are a lot of spiders at the monument They eat gnats and lots of gnats are at the monument They are attracted to the light at dusk Solution: Turn on the lights at a different time or use different kind of lights

12. Failure, Modes, and Effect Analysis Step-by-step approach for identifying all possible failures in a design, a manufacturing or assembly process, or a product or service

13. SIPOC High level process map showing suppliers, inputs, process steps, outputs and customers. It defines the process boundaries and how the parts ‘fit’ together This is important to root cause analysis in order to fully understand the process and potential causalities

14. Fishbone Diagram Identifies many possible causes for an effect or problem. It can be used to structure a brainstorming session. It immediately sorts ideas into useful categories.

15. Pareto Chart A Pareto chart is a bar graph that categories the frequency of a certain type of transaction of event. In this example of customer complaints, documents and product quality stand out. Excerpted from Nancy R. Tague’sThe Quality Toolbox, Second Edition, ASQ Quality Press, 2004, pages 376-378.

16. Root Cause Analysis and Auditor Skills

17. Question 2True or False: My team’s Internal Audit staff have the competencies and critical thinking skills to employ RCA.

18. Using RCA Daily: Employ “The 5 Cs” • Criteria – the law, regulation, contractual obligation, policy, procedure, or best practice that is expected to be followed • Condition – the factual analysis of the process as it exists • Consequence / Effect – Why the issue is important and noteworthy from a compliance, financial, or operational perspective • Cause – The root cause which allowed the condition to not mirror the criteria • Corrective Action / Recommendation – Change that will address the root Cause, allow the current Condition to reflect best practice or other Criteria, and does not cost more in relation to the Effect

19. RCA – A Basic Component of an Audit Issue Criteria: (in order of importance) Laws and Regulations Best Practices including Efficiency and Effectiveness Organization Policies and Risk Management Expectations Department Policies & Procedures Consultants Add Value By: • Creating Analysis that management does not currently have • Creating recommendations and options that management has not previously considered • Advising senior management and Board of Directors of business risks and issues they may not be aware of or wish to have independently assessed. Condition • The current status of the process/department/function • Defined by metrics of performance, compliance, profit, cost, quantitative, or qualitative • Described in perspective of operation’s/company’s cost, profit, staffing, and performance metrics Consequence/Effect/Risk: • The impact to the individual process/operation AND to the Company of having the Condition not meet the highest level of Criteria • Quantified and estimated to the extent possible Recommendation Follow Up: • Targeted review to determine whether the root cause has been corrected and condition now approximates highest level of criteria and risk management expectation • May note completion of recommendations as discussed in the audit report, other actions identified by management, no action taken because circumstances changed, or an acceptance of the risk by management. • Evaluates change in the condition Cause: The root cause for the Condition not meeting the highest level of Criteria (six sigma, or similar methodology on root cause analysis) Recommendation • Directly corrects the root cause, AND is worded to note what needs to be changed regarding the Condition • Cost of the recommendation does not exceed estimated materiality of the effect

20. Examples of what can happen when the dots do not connectTake advantage of Near misses to prevent the big misses and surprise risk events

21. JP Morgan Chase & Co. • Acknowledged a multi-billion trading loss • Specific risk management practices at fault are still under review / yet to be fully disclosed • WSJ reported on June 12th, that executives were briefed in 2010 about a foreign-exchange-options bet that went bad • Could a more robust RCA have identified governance, oversight, and policy weaknesses that would have prevented the billion dollar loss?

22. Federal Aviation Administration (FAA)Airline Near Miss Tracking • Purpose • The ASRS collects, analyzes, and responds to voluntarily submitted aviation safety incident reports in order to lessen the likelihood of aviation accidents. • ASRS data are used to: • Identify deficiencies and discrepancies in the National Aviation System (NAS) so that these can be remedied by appropriate authorities • Support policy formulation and planning for, and improvements to, the NAS • Strengthen the foundation of aviation human factors safety research. This is particularly important since it is generally conceded that over two-thirds of all aviation accidents and incidents have their roots in human performance errors. Source: US Department of Transportation Federal Aviation Administration

23. Managing Resistance and Concerns to Internal Audit Function Work on RCA • Management can be reluctant to embrace IA’s role in RCA • CAE and auditors should demonstrate the audit activity’s role and capabilities • Resistance from management in conducting RCA due to time and resource commitments • Focus on potential impacts from misses opportunities and errors versus a focus solely on likelihood • Provide both short term and long term fixes to issues • Identify near misses in your own organization that turned into larger problems as a basis for RCA • Advocate a portion of time should be spent on RCA and prioritize that effort on the biggest preventative opportunities

24. Environmental Factors of RCA • In many cases, RCA can be traced back to a person or persons • Auditors should not focus on that person/person but the environmental factors that led to that error or missed opportunity: • Competence of personnel • Hiring of qualified personnel • Lack of or insufficient training • Adequacy of technology or tools • Appropriateness of organization or departmental culture • Health, culture, morale of the organization • Level or number of resources (i.e. budget or personnel) • Process circumstances and other influencing items that led the person or persons to make the decision they made • Decision-making authority of the person or persons involved

25. Managing the Perception that RCA Places Auditors in the Role of Management • Manage this perception risk by: • Providing specific, objective, and supported analysis of the root cause • Distinguish the root cause determination from the recommendation to address root cause • Ensure the internal audit charter and engagement reporting clearly notes the role of management to assess recommendations made by internal audit and own the implementation of any changes to the process • Distinguish between engagements driven by internal audit activity that are assurance in nature versus those that are consulting and driven by the management sponsor

26. Final Thoughts • Root Cause Analysis is not an “extra” service • Rather, it is a core part of Internal Auditor’s role and insight delivery • Define the level of RCA you will undertake • “None” is simply not an option for a mature audit organization • Be prepared to sell RCA in the face of management and audit staff resistance • “Near misses” and “low probability” are insufficient rationales for avoiding RCA • Performing RCA in critical areas ensures the resiliency of your organization • Addressing the “does audit add value and insights” expectations gap requires auditors to: • take risks • demonstrate critical thinking skills • drive positive change in the organization Read the PracticeAdvisory!

27. QuestionsJames RoseCIA, CRMA, CPA, CISA, CISSP