So You Want to Audit IT?

1. So You Want to Audit IT? Harold Anderson, CPA, CIA, Internal Audit Executive, NCCI Holdings, Inc. Scott Parker, CIA, CISA, Sr. Internal Auditor, NCCI Holdings, Inc.

2. Who is NCCI? • National Council on Compensation Insurance, Inc., based in Boca Raton, FL, manages the nation’s largest database of workers compensation insurance information. NCCI analyzes industry trends, prepares workers compensation insurance rate recommendations, determines the cost of proposed legislation, and provides a variety of services and tools to maintain a healthy workers compensation system. 

3. Who is NCCI? • NCCI has operated on a not-for-profit basis since 1923. • NCCI studies workplace injuries and other national and state factors impacting workers compensation. • NCCI analyzes industry trends, prepares workers compensation insurance rate recommendations, assists in pricing proposed legislation, and provides a variety of data products to more than 900 insurance companies and nearly 40 state governments. • NCCI's core services include: • Rate and advisory loss cost filings • Cost analyses of proposed and enacted legislation • Residual market management • Production of experience ratings • Statistical and compliance services • Maintenance of the workers compensation infrastructure of classifications, rules, plans, and forms • NCCI has more than 900 employees

4. Scott’s Brush With NCCI - 1985 • Scott delivers pizza. • Scott arrives ungracefully at bottom of stairs. • X-rays are taken, invoices are generated. • Invoices are paid. • Unit Statistical Card is prepared and mailed to NCCI. • Information thereon is entered into NCCI’s system. • Information for similar injuries is compiled, and actuarial work performed. • Scott’s ungraceful moment is reflected in the 1987 Worker’s Compensation rates in Florida for class code 7380 – Drivers, Chauffeurs, Messengers, and their Helpers NOC – Commercial.

5. Scott’s Brush With NCCI - 2014 • Scott delivers pizza. • Scott arrives ungracefully at bottom of stairs. • X-rays are taken, invoices are generated. • Invoices are paid. • Unit Statistical Report is prepared and transmitted to NCCI via DTVI or SFTP. • Information thereon is edited, validated, graded, and then entered into the Integrated Database (IDB) in NCCI’s system. • Information for similar injuries is compiled, and actuarial work performed. • Scott’s ungraceful moment is reflected in the 2016 Worker’s Compensation rates in Florida for class code 7380 – Drivers, Chauffeurs, Messengers, and their Helpers NOC – Commercial.

6. What’s the Difference? BIG DATA

7. What is Big Data? • Big data refers to extremely large, complex data sets that exceed the processing capabilities of traditional IT infrastructure due to their size, format diversity, and speed of generation-The Institute of Internal Auditors • More on that later

8. So You Want to Audit IT? • Basic Steps • Select an IT Control Framework • Assess IT Risk • Determine Audit Cycle and Risk Focus • Create Audit Plan • Audits • General Controls Testing • Consulting Projects • Advisory Oversight • Execute Audit Plan • Communicate Results with Stakeholders

9. IT Control Frameworks • COBIT5(formerly COBIT4.1) • COSO • GTAG Guidelines • ISO 27002 (formerly ISO 17799) • NIST (US) • ITIL (UK) • Others?

10. Risk Assessment • Risk assessment methodologies can range from simple classifications of high, medium and low, based on the IS auditor’s judgment, to complex and apparently scientific calculations to provide a numeric risk rating. • IS auditors should consider the level of complexity and detail appropriate for the organization being audited. • Assessment should include, at a minimum, an analysis of the risks to the enterprise resulting from the loss of, and controls supporting, system availability, data integrity and confidentiality. • All risk assessment methodologies rely on subjective judgments at some point in the process (e.g., for assigning weightings to the various parameters). The IS auditor should identify the subjective decisions required to use a particular methodology and consider whether these judgments can be made and validated to an appropriate level of accuracy.

11. Risk Assessment • In deciding which is the most appropriate risk assessment methodology, IS auditors should consider such things as: • The type of information to be collected (sometimes financial effect is only measure—not always appropriate for IS audits) • The cost of software or other licenses to use the methodology • The extent to which the information required is already available • The amount of additional information required to be collected, and the cost of collecting this information (including time) • The opinions of other users of the methodology • The willingness of management to accept the methodology

12. Risk Assessment • No single risk assessment methodology can be expected to be appropriate in all situations. Conditions affecting audits may change over time. Periodically, the IS auditor should re-evaluate the appropriateness of the chosen risk assessment methodologies.

13. Risk Assessment • IS auditors should use the selected risk assessment techniques in developing the overall audit plan and in planning specific audits. • The IS auditor should consider each of the following: • Inherent risk • Control risk • Detection risk • Considerations • Disparate Infrastructure • Highly Integrated Systems • Complex Business Requirements • Buy vs. Build Decisions • Audit Fatigue

14. The Audit Plan • Standard S5 Planning states that IT audit and assurance professionals should plan the information systems (IS) audit coverage to address the audit objectives and to comply with applicable laws and professional auditing standards. They should develop and document: • A risk-based audit approach • An audit plan that details the nature and objectives, timing and extent, objectives, and resources required • An audit program and/or plan detailing the nature, timing and extent of the audit procedures required to complete the audit

15. The Audit Plan • Standard S11 Use of Risk Assessment in Audit Planning states that IT audit and assurance professionals should: • Use an appropriate risk assessment technique in developing the IT audit plan and in allocating IT audit resources • Identify and assess risks relevant to the areas under review and relationships to other auditable areas

16. The Audit Plan • Standard S12 Audit Materiality states that the IT audit and assurance professionals should consider: • Audit materiality and its relationship to audit risk to determine nature, timing, extent of procedures • Potential weaknesses or absences of controls and whether they could result in significant deficiency or a material weakness in the information system • The cumulative effect of minor control deficiencies to translate into significant deficiency in the information system

17. The IT Audit • Requirements are the same as for process audits; you know how to do those; you’re doing fine; don’t change a thing

18. Consulting Services • Consulting services are objective advisory, facilitative, and training activities, for which the nature and scope are agreed to with the customer, intended to improve governance, risk management, and control processes. Consulting engagements generally involve two parties: • Engagement Customer – the person or group seeking or receiving the advice. • Internal Audit - the person or group offering the advice. • These requests are often special requests for which no provision has been made in the annual internal audit plan. • Engagement Customer • Internal Audit

19. Types/Levels of Consulting Services • Assessment Services • Examples: Assessing the adequacy of internal control in a proposed accounts payable system, estimating the savings from outsourcing process. • Facilitation Services • Examples: Control self-assessment, benchmarking, business process reengineering support, assistance in developing performance measurement, and strategic planning support. • Remediation Services • Examples: Developing and delivering a training seminar on internal controls, drafting of policies for cash handling or writing the organization’s code of conduct

20. Benefits of Consulting Services • With the increased focus on systems of internal controls by regulators, independent auditors and management, independent auditors are being asked more frequently to train managers on the effective design, implementation, and operations of risk management methodologies and internal control activities. • Internal Audit functions that perform consulting services have great opportunities to add value to their organizations. • Improved internal audit relations with operating management.

21. Issues/Risks with Consulting Services • Balancing assurance and consulting • Crossing the limits — are there bounds on the extent of consulting a function should do? • Documentation required for the consulting engagements. • Maintaining objectivity within the Internal Audit function • Maintaining objectivity and independence responsibilities when providing assurance services following recent consulting with customers.

22. Communicating with your Stakeholders • Communicate throughout the engagement • Draft Versions of Reports • Final Engagement Communications • Distributing Formal and Informal Final Communications • Minimum Requirements • Purpose • Scope • Results • Observations • Conclusions • Recommendations • Action Plan • CAE Review • Distribution

23. IT Best Practices • Program Builds • Off-the-Shelf Configured Solutions • Mature SDLC • PMO • QA • Test Cases Tied to Business Requirements • GAP Analysis • Stakeholder Buy-In

24. Program Builds-SDLC(System Development Life Cycle) • Analysis-study, research, Business Requirements • Project Charter-Signatures • Development-Design Phase • Hardware/Software • Non-Labor • Risks to the Project-Critical Path • Build in a Contingency Reserve • Resources & Skill Sets • Project Timeline • Investment

25. Program Builds-SDLC

26. Program Builds-SDLC

27. Off-the-Shelf Configured Solutions • Can Solutions • Migration Risks • Security Assessment • Integration Testing • Databases-Default Settings • Customization • Browser Supported • Access & SOD (Segregation of Duties) • Training & SMEs (Subject Matter Experts)

28. Mature SDLC-Testing • Function Testing • Regression Testing • Data Conversion Testing • Parallel Testing • End-to-End Testing • Performance Testing • Stress, Load, Scalability Testing • Volume Testing • Security and Access Control Testing • Fail-Over and Recovery Testing • Compatibility and Configuration Testing • Beta Testing • Installation Testing

29. PMO-Project Management Office

30. Quality Assurance • What is Quality? • Conformances to requirements • To ensure software quality through the related set of validation and verification activities • Performs these specified functions correctly over repeated use or over a long period of time • Functional – Integrated, System, Regression, API, GUI Validation, Documentation • Non-Functional – Installation, Interoperability, Compatibility, Usability, Globalization, Security

31. Quality Assurance - Test Cases Tied to Business Requirements • Test Cases with Expected Results • A formal written test-case is characterized by a known input and by an expected output, which is worked out before the test is executed. The known input should test a precondition and the expected output should test a post condition. • Test for Security – Encryption, Roles & Rights • Regression – Go back and retest if significant changes have been made, especially around security.

32. Stakeholder Buy-In • Scope of the Audit • Demonstrate you know the process well enough to perform the audit – how? Flowcharts through the lens of Internal Audit (Blend System/Applications with Business Process) • Identify the controls – Obtain buy-in of what controls should be tested • Audit Test Steps • Do you have a finding – If they fix it, no question. Do they agree it’s a finding. What impact does this issue have on your process.

33. IT Audit Best Practices • Integrated Audits • Staffing Considerations • Process & IT Flow Charts

34. IT Audit Best Practices • Integrated Audits • Business Process and Application Audit • Security • General Controls as part of any Audit • Access and Segregation of Duties (SOD) • Record Retention • Backup & Restore • End User Computing Systems (EUCS)

35. IT Audit Best Practices • Staffing Considerations • Skill Sets • Previously performed the Audit • Involved as Consultant on Project Build/Enhancement • Number of High Risk Findings on Previous Audit • Do the stakeholders know you and trust you • Do we know their platform • Do we know what type of code drives the application

36. IT Audit Best Practices • Process & IT Flow Charts • Interviews and Walkthroughs • Flow Charting with Swim Lanes • Combining both Business Process with System Flow • Identify Risks and rank risks • Identify Controls • Audit from Source Data: Never Trust the Report until you’ve audited the report!

37. Back to Big Data • Big data refers to extremely large, complex data sets that exceed the processing capabilities of traditional IT infrastructure due to their size, format diversity, and speed of generation-The Institute of Internal Auditors

38. Considerations for IT Audit • Compliance with retention policies, privacy laws and data loss prevention policies • Legal and reputational risks • Skillsets & Tools • Impact on Audits

39. Know Your Data • Data Classification • Structured • Unstructured • Benefits • Reduce Costs and Administrative Overhead • Improved Data Compliance and Risk Management • Simplification of Data Encryption • Assist in effectiveness of privacy and retention policies, and definition of controls

40. Compliance with Retention Policy? • Does your company have a retention policy? • Are you ensuring that the company is retaining/omitting data according to retention policy? • Are you auditing as part of a retention cycle audit or each applicable IT audit?

41. Retention Policy Controls • Establish a Retention Policy Team • Develop a Retention Policy • Design Controls around the Retention Policy • Develop a monitoring system to determine if the control system needs changes.

42. Compliance with Privacy Laws • Privacy Laws refer to the laws that deal with the regulation of personal information about individuals, which can be collected by government, public and private organization, and its storage and use. • Health Insurance Portability and Accountability Act (HIPAA) • Financial Services Modernization Act (GLBA) • Final Rule on Privacy of Consumer Financial Information • Fair Credit Reporting Act (FCRA) • Fair Debt Collections Practices Act (FDCPA)

43. Privacy Controls • Establish a Privacy Team • Develop a Privacy Policy • Design Controls around Personal Private Info. • Develop a monitoring system to determine if the control system needs changes. • Develop Incident Response Plan • Encryption • Develop a Data Loss Prevention program with an overall Data Protection Strategy

44. Data Loss Prevention (DLP) Program Key Components • Come up with a DLP Strategy • Increase resource effectiveness • Streamline, simplify and standardize processes • Utilize technology to detect and prevent losses

45. Legal & Reputational Risks • Identify and address the ethical, legal, and reputational risks around personally identifiable customer data and sensitive intellectual property.

46. Data – Planning Activities • Identify Strategic Enterprise Data Needs • Develop & Maintain the Data Strategy • Establish the Data Management Professional Experts • Identify & Appoint Data Stewards • Establish Data Governance & Stewardship • Develop, Review & Approve Data Policies, Standards and Procedures • Review & Approve Data Architecture • Plan and Sponsor Data Management Projects & Services • Estimate Data Asset Value & Associated Data Management Costs • Develop & Maintain the Enterprise Data Model • Define & Maintain the Data Technology Architecture • Define & Maintain the Data Integration Architecture • Define & Maintain the Data Warehouse / BI Architecture

47. Data – Planning Activities • Define & Maintain Enterprise Taxonomies • Define & Maintain the Meta Data Architecture • Understand Meta Data Requirements • Define Data Quality Metrics & Service Levels • Plan for Data Recovery • Set Database Performance Service Levels • Plan for Data Retention • Understand Data Technology Requirements • Understand Data Privacy, Confidentiality & Security Needs • Define Data Privacy & Confidentiality Policies & Standards • Define Password Standards & Procedures • Understand Reference & Master Data Integration Needs

48. Data – Control (Supervisory) Activities • Supervise the Data Management Professional Staff Coordinate Data Governance Activities • Manage & Resolve Data Related Issues • Monitor & Ensure Regulatory Compliance • Monitor Conformance with Data Policies, Standards and Architecture • Oversee Data Management Projects & Services • Communicate & Promote the Value of Data Assets • Review Data Model & Database Design Quality • Manage Data Model Versioning and Integration • Implement & Maintain Database Environments • Implement & Control Database Changes • Inventory & Track Data Technology Licenses

49. Data – Control (Supervisory) Activities • Manage Users, Passwords & Group Membership • Manage Data Access Views • Manage Data Access Permissions • Monitor User Authentication & Access Behavior • Classify Information Confidentiality • Control Code Values & Other Reference Data • Archive, Retrieve & Purge Documents • Integrate Meta Data • Manage Meta Data Repositories • Distribute & Deliver Meta Data • Measure & Monitor Data Quality • Manage Data Quality Issues • Monitor Operational DQM Procedures & Performance

50. Skillsets & Tools • Big Data requires the tools, techniques and architectures for analyzing, visualizing, linking, and managing large, complex data sets. • Review your data management plan, identify the skills and tools needed to perform the task(s) • Match skills needed to available staff and tools identify gaps • Develop training/hiring plan and budget to acquire needed resources