OIA Tools & Technology Evaluation Methodology

1. OIA Tools & Technology Evaluation Methodology Prepared for State of Ohio, Office of Budget & Management Deloitte Consulting April 18, 2008

2. Content Overview

3. Objectives • Identify high level Office of Internal Audit (OIA) business needs, and technical and functional requirements for software tools • Perform high level review of existing software tools and functionality related to the OIA’s needs and requirements • Develop a software evaluation methodology, to include a criteria evaluation approach that could be used in drafting a Request for Information (RFI) to software vendors

4. OIA Technology Needs & Requirements

5. Current Needs & Requirements Per discussion with key stakeholders within OBM, OIT, and the HB166 Advisory Committee, the following high level business and technology needs and requirements were identified for OIA: • Needs: • Tools to support the management of OIA, to include time reporting, accountability, reporting and billing • Tools to support the management, maintenance, and retention of audit workpapers • Tools to support the risk assessment and audit universe maintenance function • Tools to organize and drive governance, risk, and compliance initiatives (e.g. CSA’s by IT Departments) • Tools to support the auditing of key financial and operational controls for critical application systems • Process mapping capabilities for documentation and education purposes • Continuous and automated monitoring of controls • Requirements: • Solutions must be able to support a large and distributed workforce (approx 125 auditors in various locations w/ remote capability (web-based), custom reporting, & centralized QA monitoring needs) • Solutions must be cost effective • Solutions would benefit by incorporating existing OIA tools and technology (e.g. Sharepoint, web architecture, etc.) • Solutions must have adequate security measures to safeguard audit evidence • Web-based for customer usage (e.g. comment tracking) and improved transparency for the public (e.g. final reports) • Facilitates record retention requirements & timely public record requests • Software vendor must have stability & provide adequate training/support

6. Current Vendor & Tools Landscape

7. Current Situation Analysis • Internal audit departments have growing pressure for increased oversight and assurance • With mounting pressure and increased workloads, internal audits are overwhelmed with manual and decentralized processes • Organizations are now looking to reduce the cost and improve efficiency associated with their internal audit departments • Focus has shifted from reliance on “error-prone” manual controls to automating and monitoring the execution of those controls • Interim solutions provide adequate controls repository functionality but offer little in improving the approach • Toolsare able to assist with enterprise wide risk assessment, compliance, planning, scheduling, control automation, control monitoring, review, report generation, trend analysis, and storage. • Flexibility of the software allows OIA to customize and scale the tool according to their business or changes in IT infrastructure and platforms • Tools help assess various risks (financial, environmental, health & safety, IS) under a consistent risk methodology

8. Decisions to Consider • Organizations are assessing the value/cost of implementing a tools solution to help with internal audit processes • Focus on a broader Governance, Risk and Compliance (GRC) program • Understand your current process & issues and focus on improvement • Timing, cost and level of effort to implement, to include initial and ongoing training needs, software licensing, and maintenance • How best to enhance the existing audit program into a more sustainable, repeatable process • Improve documentation version control • Consider self-assessment procedures • Improve efficiency of the process (i.e. workflow) • Enhance reporting features to improve effectiveness • Understanding the vendor landscape for the next generation of audit technologies • Find the tool that best fits your needs • GRC is the “hot” label today • Many boutique vendors in this space today to address cost concerns and unique needs

9. Software Vendors In the past, there was a clear definition between Internal Audit software and GRC software for controls auditing. Presently, these competencies are merging in new releases of software products from numerous vendors • Internal Audit Software • Focus on audit procedures and maintenance • Workpaper creation and maintenance, management signoff, audit planning and scheduling, audit budget • Centralized data repository, online checkout functionality, best practice/knowledgebase repository • Compliance with SOX • GRC Software • Focus on governance, risk, and compliance initiatives • Policy management, incident management, asset management • Risk assessment, threat management, risk dashboards • Internal audit components incorporated

10. Market Trend – Internal Auditor Software Survey • Survey respondents: • 21% Government industry • Majority of respondents from small audit shops (reason for Excel’s stronghold) • Large departments rely on specialty products (TeamMate, AutoAudit) • Software related concerns noted: • #1 – Ability to find software that meets the department’s specific needs • #2 – Cost Source: Grey, Glen. “An Array of Technology Tools.” Internal Auditor August 2006: 56-62.

11. Market Trend – Internal Auditor Software Survey Among companies who use a risk management analysis tool (beyond Excel), TeamMate (6%) and AutoAudit (3%) showed largest market share. 70% of Government agencies use audit management and risk management software tools Source: Grey, Glen. “An Array of Technology Tools.” Internal Auditor August 2006: 56-62.

12. Software Selection Methodology

13. Phase I Planning and RequirementsDefinition Phase II Request for Proposal Development and Execution Phase III Final Analysis and Recommendation Selection Methodology A three phased approach is recommended for effectively selecting a software solution: Phase I will begin by defining the scope for OBM’s tool selection process, to include business needs and requirements and setting a timeline for the process. Next, an extensive list of requirements will be reviewed and weighted according to the specific needs of OIA. A preliminary list of potential vendors will be gathered. During this phase of the selection process, the project team will take the information learned during phase I and trim the vendor list to only the most viable candidates. A request for information (RFI) will be sent to each vendor, responses will be compiled and analyzed and a demo list of 2-3 vendors will be created. Phase II will be completed by facilitating the vendor demonstration process, scoring and compiling of results. During the final phase of the selection process, the project team finalizes the selection process, presents the compliance system recommendation to executive management and facilitates next steps toward solution implementation.

14. Sample Timeline / Milestones

15. Tool Selection Process Phase I Planning and Requirements Definition Phase II Request for Proposal Development and Execution Phase III Final Analysis and Recommendation Selection Process – Phase I Tasks • Initiate Project and Establish Team Roles and Responsibilities • Establish project objectives, scope, priorities, and determine key milestones. • Assist OIA in identifying project team members, formalize team structure and reporting responsibilities, and develop detailed project work plan. • Identify Unique Functional and Technical Requirements • Develop the unique business and system requirements for the software tools. These requirements will be used in Phase II as a basis for determining which tool provides OIA the best fit to their requirements. • Establish Critical Success Factors • Meetings focused on communicating and affirming issues and critical success factors, understanding specific project expectations, and identifying how these will impact OIA’s organization and the selection project. • Functional and technical requirements coupled with the critical success factors will serve as a detailed checklist to guide and facilitate vendor demonstrations. Key Deliverables • Detailed Project Work Plan • Defined and Weighted Functional and Technical Requirements List

16. Requirements • Weight business and technical requirements according to OIA’s needs

17. Tool Selection Process Phase I Planning and Requirements Definition Phase II Request for Proposal Development and Execution Phase III Final Analysis and Recommendation Selection Process – Phase II Tasks • Develop Short List of Vendor Candidates • Gather knowledge within the marketplace to identify a short list of 4-5 potential candidates. • Develop RFI/Scorecard and Solicit Vendor Bids • The Request for Information (RFI) will require each vendor to provide a sample implementation schedule, pricing, warranty, references and other pertinent guidelines for the bidders to follow. OIA to populate the vendor scorecard. • Vendors will be contacted and bids solicited from them. • Facilitate Comparative Analysis of RFI Responses • Review requirements and institute a ranking system to evaluate the vendor proposals. • Collect vendor RFI responses and prepare a comparative analysis report. • Utilize comparative analysis report to further condense candidate list to 2-3 vendors for demonstration. • Facilitate Vendor Demonstration Sessions • Invite top candidates to OIA to present their system and to answer/clarify specific questions related to their RFI response. • Develop demo scripts to evaluate vendors Key Deliverables • Short List of Vendor Candidates • Request for Information (RFI) • Vendor Scorecard • Comparative Analysis Report

18. Request for Information Sections: • Company (OIA) background and overview, description of desired solution, benefits sought • Response directions - OIA contact information, response due date, target demo dates • Vendor information – company profile and tool, including financial solvency & market share • Requirements – customized questions according to OIA requirements, response from vendor limited to 500 characters. • Customer references – list of customers of similar size/requirements • Disclaimer – RFI solely for informational and planning purposes

19. Tool Selection Process Phase I Planning and Requirements Definition Phase II Request for Proposal Development and Execution Phase III Final Analysis and Recommendation Sample System Requirements Selection Process – Phase III Tasks • Facilitate Vendor Reference Checks • Utilize a customized questionnaire to evaluate each vendor’s performance at comparable clients. The questionnaire will be customized to OIA’s specific requirements and interests. • Prepare Final Selection & Recommendation Report • The project team will work together to put the finishing touches on the selection report. • Provide guidance for executive-level presentation. • Select the Best Internal Audit Tool (or Tools) • Conduct a detailed review session to reach agreement that OIA’s requirements are met by the chosen vendors and solutions. • Finalize Selection & Recommendation Report • Provide recommendation report template for OIA to complete and present. • Support OIA in Vendor Contract Negotiations • Present OIA with applicable rate and licensing information based on existing vendor relationships. Key Deliverables • Reference Check Summary • Final Business Case