A Delphi Study into the Audit Challenges of IT Governance in the Australian Public Sector

1. Group 3 Claire ZHANG Max MA LAI YANYAN A Delphi Study into the Audit Challenges of IT Governance in the Australian Public Sector L. AL Omari, PH Barnes and G. Pitman

2. INTRODUCTION As this research is categorized as applied research, the research scope will be narrowed down on multiple aspects in order to maintain a sufficient level of internal validity.

3. Definitions and Concepts Audit: Audit is a discipline born of the need to assess the degree of conformation with standard practice and addresses a wide range of assurance and consulting services through the utilisation of methodologies and frameworks. IT Governance: IT governance (ITG) is defined as the processes that ensure the effective and efficient use of IT in enabling an organization to achieve its goals.

4. b. Research Objective What: The aim of this research is to provide insight into the range of IT governance audit challenges currently facing Queensland public sector organizations and likely future challenges.

5. b. Research Objective 2. Why: 1) IT is playing an important role. 2)ITG consists of structures, processes and relational operational mechanisms working together as one entity to ensure that IT and business objectives are aligned. 3)Audit provide a credible level of assurance of rigour in ITG by systematically examining controls efficiency, identify key risk areas, advise about possible IT failures, as well as offer suggestions on how to improve current practices( not only the traditional function as being a corporate watchdog)

6. b. Research Objective 2. Why: 4)In Australia, the public sector regularly relies on IT to deliver quality and accessible services to the community, such as E-Government services.Public sector entities are under increasing pressure to exhibit transparency and accountability in using taxpayers’ money to deliver outcomes at the same time as operating under greater budgetary constraints, higher complex regulatory requirements, and struggling to attract staff when compared to the private sector

7. There is two key questions that built the research What are the significant challenges and why they are important? What are the perceived top ten IT governance audit challenges in the Queensland public sector?

8. Research Method

9. Researchmethod Criticalreviewofliterature Informalsmallgroupdiscussion Delphiresearch ->triangulationofdatacollectionsourcesinordertogaincompleteunderstandingofauditchallengesinITG

10. Delphi method The objective of most Delphi applications is the reliable and creative exploration of ideas or the production of suitable information for decision making. The Delphi Method is based on a structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires interspersed with controlled opinion feedback (Adler and Ziglio, 1996) Baldwin (1975) asserts that lacking full scientific knowledge, decision-makers have to rely on expert opinion. The Delphi method has been widely used to generate forecasts in technology, education, and other fields (Cornish, 1977).

12. Researchscope Tomaintaininternalvalidity 1)restrictedtopublicsector 2)reducedingeographictermsandtookintoaccountthesizeoforganizations 3)onlyfocusonQueenslandpublicsectororganisationswithheadcountsrangingfrom100tomorethan1000employees. 4)itonlyfocusedonperformanceaudit

13. Research questionnaire Type of questionnaires: Email survey Participants: 28 experts of the Queensland Public Sector Beforesentingoutthequestionnaires:thereispilot-testedonfiveexperts(partitionersandacademics)forambiguitiesandvaguenesspriortoadministeringtopanelmembers Thequestionnaire willprovidecleardefinitionoftheterms

14. Research questionnaire Three-round questionnaires instrument with decreasing number of experts ( from the initial group, 24 experts continued to be inthesecond round, 20 experts continued to be in the third round and 16 experts involved in the full research effort)

15. Research questionnaire Three-round of questionnaires • First stage: the respondents get a preceded initial list of ITG audit challenges and asked them to validate its suitability to public sector. • Second stage: Respondents were asked to rate on a 5-point scale for each of the revised ITG audit challenges the “perceived impact”(0=noimpact,5=highimpact),and the “perceived effort to address”(0=noeffort, 5=higheffort). Then, theywereaskedtotake thepreviousattributesofimpact,efforttoaddressandpersonalexperienceinto account to provide theirperception of top ten ITGauditchallenges. • Third stage: thepanellistswere asked to re-evaluate their round two ratings, taking thegroup averagesinto consideration. (goal:getagreaterconsensuswithinthegroup)

16. Sampling techniques and statistical methods Non-Probability sampling: Research questions and objectives that do not require such generalizations

17. Sampling techniques and statistical methods Non-Probability sampling techniques: Purposive: as the research had its purpose to find out the challenges of the Public sectors, so the respondents we select should be the experts of Queensland In this case, it stated that to ensure the quality and accuracy of data, special attentions was given to selecting qualified panel experts. It is forecasted that 10 to 15 participants may be adequate for a focused Delphi.

18. Sampling techniques and statistical methods ThedegreeofconsensusatthefinalroundwasmeasuredleveragingKendall’sWcoefficientscale,specificallyforthequestiononthetop-10ITGauditchallenges.

19. Results and interpretation Different challenges and issues were identified based on literature research and discussion with informants.Consequently, three categories were created, namely, Internal (N),External (E), and Organizational (O) audit challenges.

21. Results discussion Round one- Validating the Initial list All data was structured and analyzed resulting in an extended list of challenges. Based on this round, an updated list of challenges was used as basis to start up rounds 2 and 3.

22. Results discussion Round 2 and 3-Evaluating ITG Audit challenges. The dominance of organizational challenges is clear as they occupy four out of the top five ranks for impact and required effort. The results of this research raise questions on how public sector can increase the board’s involvement in practice.

23. Results discussion Round 2 and 3-Evaluating ITG Audit challenges. “quick wins” is defined as an audit challenge that is considerably high in impact and generally requires minimal effort that can be implemented in a short period of time, or requires reduced resources in a timely manner and cost effective In this journal they all belong to the internal audit category, and focus on the audit team involvement in ITG audit.

25. Interpretation • Average responses for impact and effort for internal, external and organizational challenges.

26. Interpretation Indicating that organizational and internal audit challenges are in general perceived as having a higher impact on the public sector than external audit challenges It appears that internal and external audit challenges are perceived as being easier to address compared to organizational audit challenges.

28. Key factors rating analysis

29. Interpretation The majority of the challenges have high impact and are difficult to implement. Just as in literature, there is a growing focus on risk-based audit approach and recognizing differences in the nature of business instead of the traditional one-size –fit-all controls testing approach. In addition, the scope of audit has expanded to include the evaluation of the effectiveness of governance processes. To that understand the unique risks associated with each different organization being audited.

30. Conclusion What are the significant challenges and why they are important? What are the perceived top ten IT governance audit challenges in the Queensland public sector?

31. For the first question: • The research identifies a list of 30 ITG audit challenges at level of executive/senior business and IT management. The results demonstrate that higher impact and/or easier to address than others. • Some challenges are perceived as not having a high impact while others are perceived as not easy to address in the context of ITG audit. • These challenges are less likely to come across in the conducting of ITG audit.

32. The research also assessed key factors derived from the Delphi analysis based on background, role, and years of experience to explore and highlight the effect of these factors on the perception of ITG audit challenges. It was observed that different key factors have different and sometime conflicting opinions.

33. For the second question: • This paper also brought up a list of top ten ITG audit challenges, specifically for the Queensland public sector in an effort to answer the second research question. This suggests that, in performing ITG audit within a public sector organization, these challenges may play an important role in preventing a successful outcome (inhibiting factors). Of course, they should be supplemented with other challenges as required by the specific environment of the organisation, to create a specific set or subset of ITG audit challenges.

34. THANKS Q&A