Conducting high quality behavioral research in auditing

1. Conducting high quality behavioral research in auditing Kathryn Kadous 2011 Auditing Doctoral Consortium

2. Some broad principles for conducting high quality behavioral research • Identify important topics that you have something to say about. • Test theory, not implementation. • Pay special attention to good experimental design. • Write like a scientist.

3. Identify important topics that you have something to say about. • Choose a topic that is consequential. • Map it into a testable theory.

4. Choose a topic that is consequential. • Some audit topics have staying power. • What do more experienced auditors know that less experienced auditors don’t know? • How can we make auditors better at fraud detection? • How will a change in financial reporting standards/audit standards/form of audit report influence auditor behavior/performance? • Others are more recent • How do client-auditor interactions influence financial reports? • What do more experienced or specialist auditors behave differently than less experienced or non-specialist auditors? • Under what conditions will auditors take advice from other auditors? • How does systems thinking/strategic thinking/brainstorming influence auditor performance? • How do auditors resolve conflicting accountabilities?

5. Map the topic into testable theory. • Psychology is a rich source of theory regarding individual and group behavior. • Management/organizational behavior has theory relevant to individual behavior in a social setting. • Sociology, economics also relevant. • Auditing frameworks summarize theory as translated into the relevant auditing setting.

6. Test theory, not implementation • Identify the core constructs. • Focus on the theoretical mechanisms. • Think deeply about the constructs and how they affect one another.

7. Identify core constructs • RQ: How will companies’ use of IFRS versus GAAP influence the quality of financial reports?

8. Identify core constructs • Once we have core constructs we can: • Assess whether these constructs theoretically related to judgment quality. • Choose/modify/create appropriate theory. • Create better designs/manipulations.

9. Focus on theoretical mechanisms • Theories (unlike “empirical questions”) can be generalized. • Theories give clues about conditions under which judgment is better/worse. • Theories help to identify how judgment can be improved.

10. Focus on theoretical mechanisms • Once we have identified theoretical mechanisms we can: • Design appropriate tests for these mechanisms. • Identify moderators and boundary conditions.

11. Think deeply about the constructs and how they affect one another • Given the theory, what additional constructs are likely to intervene in the process? • These are also at the construct level. • And are relevant to the accounting setting. • Once we have done the deep thinking we can: • Design more interesting and relevant experiments. • Design experiments that are more likely to be successful.

12. How can I improve my ability to test theory? • Read, a lot! • Read frameworks, think pieces in accounting • Nelson 2009 AJPT—Framework for classifying failures in professional skepticism • Peecher, Solomon, Trotman 2010 WP—How to resolve auditors’ conflicting accountabilities • Hammersley 2010 WP—Fraud planning performance • Read outside of accounting. • Take seminars outside of accounting. • Attend conferences, brown bags, talks outside of accounting. • Sign up for TOC services.

13. Pay special attention to good experimental design • Hold “all else” equal. • Fully cross designs when possible. • Choose experimental tasks that match the “deep structure” of the one you are interested in.

14. Holding “all else” equal • Not always easy, but critical for testing theory. • Identifying constructs is half the battle. • Sometimes additional complications require pre-tests, or other creative solutions.

15. Fully crossing designs • Is not always necessary/possible, but typically is desirable. • Avoids need for diagonal comparisons or confounded comparisons. • Allows clear, easy tests of effects, including differential simple main effects.

16. Choosing experimental tasks • Important that participants go through the relevant part of the decision process they would in the real task. • Not necessarily important that task looks the same, responses taken in same format, etc.

17. How can I improve my designs? • Read, a lot! … etc. • Take time to get it right. • Take advantage of brown bags and other informal opportunities to present. • Enlist help of your colleagues. • Run ideas by faculty. • Collect data only when you are sure that the data you get will allow you to test your hypotheses.

18. Write like a scientist to give your paper its best shot • Theory section should describe mechanisms, not just prior research. • Relevant prior research should be clearly described and given its due, and your contribution clearly identified. • Being first is not a sufficient contribution! • Hypotheses should be testable. Comparisons should be clear. • Tests should be appropriate for the form of the data. • Writing should be clear. • Once you are happy with it, get feedback! • Then submit only when you have taken advantage of feedback and made the paper the best it can be.

19. Thanks for listening! “Real success is finding your lifework in the work that you love.” • David McCullough