Operational Auditing Tools for Improving Business Outcomes

1. Operational Auditing Tools for Improving Business Outcomes Mike Jacka, CIA, CPA mike_jacka@farmersinsurance.com Paulette Keller, CIA, CPA paulette.keller@farmersinsurance.com

2. Outline Definitions & Standards Relationship with Management Operational Auditing Tools Risk/Control Matrix Business Process Mapping RACI Matrix Customer Mapping Spaghetti Maps

3. Internal Audit - definition IIA definition of Internal Auditing � �Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization�s operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes.� Why operational auditing � We can first look at what the standards tell us what our general responsibilities are. We can start with the overall definition of internal auditing. What are the key words you see that help direct us. Independent/objective Assurance/consulting Add value Improve operations Systematic approach Improve effectiveness Why operational auditing � We can first look at what the standards tell us what our general responsibilities are. We can start with the overall definition of internal auditing. What are the key words you see that help direct us. Independent/objective Assurance/consulting Add value Improve operations Systematic approach Improve effectiveness

4. Evaluating Controls Standard 2130 � Control The internal audit activity should assist the organization in maintaining effective controls by evaluating their effectiveness and efficiency and by promoting continuous improvement. 2130.A1 - The internal audit activity must evaluate the adequacy and effectiveness of controls in responding to risks within the organization�s governance, operations, and information systems regarding the: Reliability and integrity of financial and operational information; Effectiveness and efficiency of operations; Safeguarding of assets; and Compliance with laws, regulation, and contracts. 2130.A2 - Internal auditors should ascertain the extent to which operating and program goals and objectives have been established and conform to those of the organization. 2130.A3 - Internal auditors should review operations and programs to ascertain the extent to which results are consistent with established goals and objectives to determine whether operations and programs are being implemented or performed as intended. Read this standard. When we look at Standard 2130 on Evaluating Controls � what do you think is important about this main definition. Evaluate effectiveness and efficiency Promoting continuous improvement What about the detailed guidance? A.1 � with respect to organizations governance, operations, and info systems evaluation of efficiency and effectiveness of operations included A.2 � existence of goals and objectives/consistency with organization A.3 � evaluation results, consistency with goals/objectives to determine if implemented and performing as intended. Read this standard. When we look at Standard 2130 on Evaluating Controls � what do you think is important about this main definition. Evaluate effectiveness and efficiency Promoting continuous improvement What about the detailed guidance? A.1 � with respect to organizations governance, operations, and info systems evaluation of efficiency and effectiveness of operations included A.2 � existence of goals and objectives/consistency with organization A.3 � evaluation results, consistency with goals/objectives to determine if implemented and performing as intended.

5. Operational Auditing - definition A systematic check or assessment of the efficiency or effectiveness of a company�s business operations The objective of operational auditing is to appraise the effectiveness and efficiency of a division, activity, or operation of the entity in meeting organizational objectives and goals Discussion regarding understanding of what operational audits are. Evaluation of business operations � looking at all aspects of business operations, not just focused on financial reporting or compliance issues. Looking at efficiency and effectiveness � we do not look only at control activities to assure they are operating as management intended, we look at the efficiency and effectiveness of the operations. Could the entire operation be improved by streamlining some activities, or by working more effectively or economically? Is the operation meeting management�s goals and objectives? What are the goals and objectives for the particular operation? Operational auditing requires us to look at the business through management�s eyes. It will require us gain a good understanding of the business, and to work closely with both management and people performing the work to determine if improvements in efficiency or effectiveness can be made. We will look at more than just policies, procedures, and control activities. We will look at the entire framework, including people, risks, communication and monitoring activities and how they relate to the accomplishment of objectives. We may even generate innovative and creative ideas for improvement while performing an operational audit. But, we must take care that be maintain our independence. Discussion regarding understanding of what operational audits are. Evaluation of business operations � looking at all aspects of business operations, not just focused on financial reporting or compliance issues. Looking at efficiency and effectiveness � we do not look only at control activities to assure they are operating as management intended, we look at the efficiency and effectiveness of the operations. Could the entire operation be improved by streamlining some activities, or by working more effectively or economically? Is the operation meeting management�s goals and objectives? What are the goals and objectives for the particular operation? Operational auditing requires us to look at the business through management�s eyes. It will require us gain a good understanding of the business, and to work closely with both management and people performing the work to determine if improvements in efficiency or effectiveness can be made. We will look at more than just policies, procedures, and control activities. We will look at the entire framework, including people, risks, communication and monitoring activities and how they relate to the accomplishment of objectives. We may even generate innovative and creative ideas for improvement while performing an operational audit. But, we must take care that be maintain our independence.

6. Maintaining Independence Standard 1100 � Independence and Objectivity The internal audit activity should be independent, and internal auditors should be objective in performing their work. Standard 1120 � Individual Objectivity Internal Auditors should have an impartial, unbiased attitude and avoid conflicts of interest. What does independence mean? Independence - if can carry out work freely and objectively. Obtained by organizational status What is objectivity? Objectivity is a state of mind � not subordinating judgment to others, avoid conflicts of interest � or if one does arise address it appropriately. What does independence mean? Independence - if can carry out work freely and objectively. Obtained by organizational status What is objectivity? Objectivity is a state of mind � not subordinating judgment to others, avoid conflicts of interest � or if one does arise address it appropriately.

7. Relationship with Management Why does management need operational audits? What type of relationship will Audit need with Management to facilitate a review of operations? What kind of competencies do auditors need to perform operational reviews? Why does management need operational audits? Busy running the business � don�t have time to step back and evaluate Need an objective opinion What type of relationship will Audit need with Management to facilitate a review of operations? Partnership What kind of competencies do auditors need? Knowledge of the business. Now we will look at some tools that can help us in our evaluation of the efficiency and effectiveness of operations. Why does management need operational audits? Busy running the business � don�t have time to step back and evaluate Need an objective opinion What type of relationship will Audit need with Management to facilitate a review of operations? Partnership What kind of competencies do auditors need? Knowledge of the business. Now we will look at some tools that can help us in our evaluation of the efficiency and effectiveness of operations.

8. Operational Auditing Process Identify management�s objectives Determine where an operational review would add the most value Gain an understanding of the process Prepare a risk/control matrix for the areas under review Assess efficiency and effectiveness of the control design Assess the efficiency and effectiveness of critical controls Report findings and/or recommendations

9. Operational Auditing Tools Risk Control Matrix Business Process Mapping RACI Matrix Customer Mapping Spaghetti Maps

10. Risk Control Matrix Start with management objectives (What do they want to achieve � the end result) Identify risks to accomplishing those objectives (What could go wrong?) Identify controls for each risk (What could prevent the risk or minimize it?) It is a tool that provides a systematic process for assessing business objectives, risks, and controls. Risks and controls do not stand alone � Risks relate to some objective; controls relate to some risk (if not why are they there).It is a tool that provides a systematic process for assessing business objectives, risks, and controls. Risks and controls do not stand alone � Risks relate to some objective; controls relate to some risk (if not why are they there).

11. Identifying objectives Objectives state what is trying to be accomplished, not how Strategic objectives What must be done to assure the organization�s strategy is achieved. Process objectives What the process is trying to achieve (why it exists) Broad objectives and sub-objectives may exist While strategic objective are generally well defined and easy to access, process objectives may never have been specifically defined. Each process will have an overriding objective, and will often have sub objectives as well. Processes and sub processes exist for a reason � we need to determine what the process is trying to accomplish and how it supports the strategic objective. The process objective should represent the important things the process is meant to accomplish What � not How While strategic objective are generally well defined and easy to access, process objectives may never have been specifically defined. Each process will have an overriding objective, and will often have sub objectives as well. Processes and sub processes exist for a reason � we need to determine what the process is trying to accomplish and how it supports the strategic objective. The process objective should represent the important things the process is meant to accomplish What � not How

12. Exercise 1 � Objective Evaluation The organization has a strategic objective of assuring the right people are in right place at the right time. HR is a support function that assists management in execution of the organization�s people management strategies and it also helps assist the organization in compliance with legal and regulatory requirements. Key processes HR is involved with include: terminations, recruiting, promotions, performance review, and succession planning. Determine if the following objectives are good objectives for HR. If not, explain what is wrong with the objective. HAND OUT EXERCISE 1 Individual ExerciseHAND OUT EXERCISE 1 Individual Exercise

13. Exercise 2 � Car Stereo Objective Setting You work for the Good, Cheap Car Stereo Store. The store is a local chain with 4 stores in the metropolitan area. There is a showroom, where the sales staff greets potential customers and shows them products. If a customer purchases a stereo, they can install it themselves, or they may have it installed at the store. An inventory of stereos and installation materials is maintained at the local store. Write objectives for the following processes for Good, Cheap Car Stereo Store. Make any assumptions you need to. Sales Stocking Installation Customer Satisfaction Break into small groups Hand Out Exercise 2 � have them work on the objectives Share objectives and pass out suggested solutions. Break into small groups Hand Out Exercise 2 � have them work on the objectives Share objectives and pass out suggested solutions.

14. Identifying Risk Risks are related to objectives Risks are things that could happen that might impact the achievement of business objectives. Risks are not the opposite of the objective. Risks are events. You can also think of what needs to go right to assure the objective is achieved � if it does not go right, then you have your risk.

15. Exercise 3 � Identifying Risk Look at the objectives developed in the Car Stereo exercise. Identify two key risks for each objective. Use any personal knowledge you have about stereos, shopping, or sales. Keep in same groups � identify risks.Keep in same groups � identify risks.

16. Risk Control Matrix Possible forms for matrix � Handout Practice using a Risk Control Matrix � Exercise 4 A Risk Control matrix can take on may forms There is no one �right� matrix The matrix provide an systematic method to document the objectives, risks, and controls, to rank them, and in some cases to provide a preliminary assessment of them. Some people link these objectives to COSO or the COSO ERM framework. Handout � Risk/Control Matrix Examples � discuss Any of you use another form? Handout Exercise 4 � Explain - Have work in teams to complete matrix, A Risk Control matrix can take on may forms There is no one �right� matrix The matrix provide an systematic method to document the objectives, risks, and controls, to rank them, and in some cases to provide a preliminary assessment of them. Some people link these objectives to COSO or the COSO ERM framework. Handout � Risk/Control Matrix Examples � discuss Any of you use another form? Handout Exercise 4 � Explain - Have work in teams to complete matrix,

17. Business Process Mapping A holistic approach

18. Business Process Mapping Why analyze processes? The accumulation of activities that take place in each business process is what ultimately determines whether or not a business meets its objectives.

19. Benefits of Business Process Mapping Holistic view Employee�s buy-in Sense of pride Customer driven

20. Process defined Input Action Output Feedback

21. The process of mapping Process identification Data gathering Interview and map generation Analyzing the data Presentation

22. Process identification What you see is primarily what you look at�� Determine where the process begins and ends Major events that trigger the start and end of the process Naming the major processes Process timelines

23. Data gathering Where you go to learn it Process owners Unit owners Unit owners or employees What you don�t know Process description overview, unit owners, objectives, risks, measures of success. Process definition � detailed, Triggers, inputs/outputs Tasks

24. Data Gathering Tool

25. Interviewing and Mapping Ground rules Buy-in, time, privacy, tone, listen Mapper and recorder roles Select the right people

26. The Map Supplies Sticky notes, felt tip pens, poster sized sticky paper. People in columns � chronologically Time � top to bottom Action � verb/noun on post-it Basic symbols Cycle times

28. Hands On Exercise 5 � Process Mapping Basics Exercise 6 � Process Mapping Basic Symbols Exercise 7 � Process Mapping Verb/Noun Format Exercise 8 � Process Mapping Complete Map

29. Into the editing room Inputs, outputs, and waste Surprises Invisible consequences Process ownership Objectives, risks, controls Measures of success

30. Analysis tips Remove approvals Looping errors Isolate delays, rework, handoffs Follow the forms Incomplete maps � dangling actions, unanswered questions Question hold files Look at cycle times

31. Finalize the project Look for areas to add value Business value � improve efficiency, effectiveness, or provide economies Customer value Present conclusions and recommendations

32. Pitfalls and traps We�re drowning in information and starving for knowledge Mapping for mapping sake Round and round � up and down Failure to finalize Letting the clients define the process Leading the witness Verifying the facts Do not forget the customer No tool is an island

33. RACI Matrix Responsibility Charting Sometimes problems may occur in a process because responsibilities are not clearly defined, or because too many people are involved in a process when they really do not need to be. Use of a RACI Matrix can help you to identify how problems with responsibility may be adversely impacting the process. Sometimes problems may occur in a process because responsibilities are not clearly defined, or because too many people are involved in a process when they really do not need to be. Use of a RACI Matrix can help you to identify how problems with responsibility may be adversely impacting the process.

34. RACI Matrix � Responsibility Charting A technique for identifying functional areas, key activities, and decision point where ambiguities exist. Management actively participates in the process of systematically describing activities, decisions that have to be accomplished, and clarification of the responsibility that each plays in relation to those activities and decisions.

35. RACI - Benefits Assists work teams in charting roles and responsibilities Clarifies individual/departmental roles and responsibilities Identifies accountabilities Eliminates misunderstandings, encourages teamwork Reduces duplication of efforts Establishes �consults� and �informs� resulting in better communication misunderstanding

36. RACI Matrix - Guidelines Look for improved efficiency and economies Eliminate �checkers checking checkers� 100% accuracy not always required Place accountability (A) and responsibility (R) at the level closest to the action or knowledge There can be only one accountability per activity Authority must accompany accountability Minimize the number of consultants (C) and informs (I) All roles and responsibilities must be documented and communicated.

37. RACI Matrix - Explainer �A� � Accountable � �The buck stops here� � position with yes/no authority. Only one A per function. �R� � Responsible � �The doer� � Position working on the activity. Responsible for action/implementation. Degree of responsibility determined by individual with the �A�. �C� � Consult � �In the loop� � Position involved prior to decisions or action - two-way communication. �I� � Inform � �Keep in the picture� � Position that needs to know of the decision or action � informed after � incorporates one-way communication.

38. RACI � Determining Responsibilities Exercise 9

39. RACI Matrix Development Process Develop decision /activity list - (tasks � action verbs) Develop function lists (position assigned to an activity or sub-activity) Agree on function definitions Output in a responsibility chart Agree chart with key individuals Circulate

40. RACI Chart Example

41. RACI � Vertical Analysis

42. RACI � Horizontal Analysis

43. RACI Matrix - Practice Exercise 10 � Expense Reporting RACI Matrix

44. Customer Mapping �There is only one valid definition of a business purpose: to create a customer.� Peter Drucker � the Practice of Management (1954)

45. Customer MappingIdentifying Jobs the Customer Wants to Get Done A business is �defined by the want the customer satisfies when she buys a product or service� Peter Drucker

46. Identifying customer jobs When the business focuses on the jobs the customer is trying to get done, they can develop innovative solutions to satisfy those jobs. Asking customers what they want is not necessarily going to provide you with a good definition of the job they want to do. When Apple developed the Ipod, they defined the job the customer�s wanted done as �managing their music�, not a better music player. The focus is on the job � not the solution.

47. Customer Mapping Process Mapping focuses on the Company�s Processes � but what if those processes are not producing outcomes the customer desires? Customer Mapping focuses on the job the customer is doing, the tasks the customer must do to accomplish the job, and the customer�s measures of success.

48. Customer Mapping � the process Define the job Define the tasks the customer must perform (verb/noun) Define customer inputs and outputs Define the customers measures of success (minimize/maximize) Rank the measures of success Compare results to the measures Look in depth (Process Maps) for areas that do not measure up Exercise 11 � Group Exercise - Complete a Customer Profile Worksheet

49. Customer Profile Worksheet

50. Spaghetti Map �As Is� and �To Be�

51. Spaghetti Maps While traditional flow charts or process maps may be used to identify control points, spaghetti diagrams can be used to quickly identify inefficiencies. Interview employees to determine the process flow. Document the �As Is� process flow using a narrative and nodes to symbolize the movement of key items through the process. Document the �To Be� process by eliminating some of the steps in the process which might create efficiencies.

52. Spaghetti Map - Example

53. Spaghetti Map � To Be Exercise 12 Create to �To-Be� Spaghetti Map of the Employee Expense Reimbursement process using everything you have learned about this process or information you know about other processes that are more efficient.