1 / 27

ABO Doctoral Consortium

ABO Doctoral Consortium. Choosing Participants for Financial Accounting Experiments October 10, 2008. Objectives. Identify participants in financial accounting experiments Identify characteristics of participants Discuss factors to consider when choosing participants

crwys
Download Presentation

ABO Doctoral Consortium

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ABO Doctoral Consortium Choosing Participants for Financial Accounting Experiments October 10, 2008

  2. Objectives • Identify participants in financial accounting experiments • Identify characteristics of participants • Discuss factors to consider when choosing participants • Provide advice for finding participants

  3. Financial Reporting Process

  4. Research Process

  5. Variables Influencing Judgments and Decisions • Decision-Maker Characteristics • Task Characteristics • Environmental Characteristics (Sarah Bonner, Judgment and Decision Making in Accounting, Prentice-Hall, 2007)

  6. Participant (Decision-Maker) Characteristics • External / Easily observable or measured • Gender, Age, Professional Certifications, Work Experience • Internal / Need to be measured via test or possibly manipulated • Abilities • Knowledge • Preferences

  7. Participant Characteristics - Abilities • Abilities • Verbal abilities – reading comprehension • Reasoning abilities - interpret data, reason from analogy, reason analytically • Spatial abilities – identify and comprehend patterns • Measurement • Typically through standardized college admission tests – SAT and GMAT • Manipulation • Difficult in adults

  8. Participant Characteristics - Knowledge • Knowledge • Semantic • General world knowledge, domain-specific knowledge, sub-specialty knowledge • Episodic knowledge/beliefs based on personal experience • Incentives, reactions of other parties, empirical regularities • Measurement • Knowledge-specific tests (e.g., CPA exam questions) • Post-experimental questions about beliefs • Manipulation • Possible, but need to be careful

  9. Participant Characteristics - Preferences • Preferences (Utility) • Money (Risk Aversion) • Effort • Honesty, ethical behavior, welfare of others • Measurement • Validated tests from economics, psychology, sociology • Manipulation • Induced preferences for money (Berg et al. QJE 1986)

  10. Role of Participant Characteristics • Independent Variable • Decision-maker characteristic directly affects dependent variable • McDaniel et al. (TAR 2002) • Moderating Variable • Decision-maker characteristic changes the relation between a task or environmental variable and the dependent variable • Elliott (TAR 2006) • Neither of above – just choosing participants for your task • Think in terms of a moderating role

  11. X Participant Characteristics as Independent Variable (McDaniel et al.) Causal relation Y Experience (Accounting) (Business) Judgment (Reporting Issue Identified – recurring vs. nonrecurring)

  12. Participant Characteristics as Independent Variable • Not typical in financial accounting research • Typically requires measuring participant characteristics – external/internal • May be hard to avoid confounding • External characteristics (e.g., years of experience) typically represent a number of internal characteristics (e.g., knowledge, ability) • For example, knowledge and ability are rarely independent (Link 2 in Libby and Luft AOS 1993) • A problem if you want to establish causality for internal characteristics (need manipulation + random assignment, not measurement)

  13. Participant Characteristics as Moderating Variables (Elliott 2006) U Knowledge Y X GAAP – ProForma Reconciliation (Yes/No) Decision (Investment Decision)

  14. Participant Characteristics as Moderating Variables • Use this approach even if you aren’t planning on a moderating variable approach • Does theory suggest that judgments and decisions will depend on characteristics (abilities, knowledge, preferences) of participants? Will the JDM process or outcomes change? • Does the level of relevant characteristics vary among relevant decision-makers or participant groups?

  15. Participant Characteristics as Moderating Variables • If answers to prior questions are yes, how do I proceed? • Embed theory in paper and test multiple groups that should differ based on theory – true moderating study • Examine only one group • What group is most important in terms of the research question? • External validity limitations

  16. Participant Characteristics – Professionals versus Students • Some generalizations • Basic reasoning abilities typically don’t change much after adulthood • College students likely have similar abilities to most adults • Be careful – there can be cross-sectional variation - top people in any field have survived tournaments and are likely to have higher ability

  17. Participant Characteristics – Professionals versus Students • Some generalizations • Knowledge is a big factor in accounting experiments • Semantic knowledge – world knowledge and sub-specialty knowledge increases with experience • Episodic Knowledge / Beliefs also change with experience – personal experiences with incentives, behavior of other people

  18. Participant Characteristics – Professionals versus Students • Some generalizations • Preferences • Reaction to economic incentives is basic human reaction • Experimental research based on economic theory typically can use abstract tasks and college students

  19. Issues – Working with Professional Participants • Negative externalities of using professionals when not needed: limited resource (Libby, Bloomfield & Nelson AOS 2002) • Places high demands on researcher – good research question requiring professionals, good design, good instrument • Personal experiences of professionals lead to expectations that will create problems if not met • Need to run materials by professionals and pilot test

  20. Issues – Working with Professional Participants • Selection and non-response bias may be stronger with professionals than with students (Peecher and Solomon 2001) • End up using one firm – how general are your results? • Professionals are busy people which leads to non-response • Need to get them interested and committed • Work around their schedule

  21. Source of Participants • Preparers • Alumni directory – controllers, CFO’s • Financial Executive Institute • Institute of Management Accountants

  22. Source of Participants • Investors - Nonprofessional • College Students • Think carefully about research question and task • Insights on using MBA students as proxies for investors – Elliott et al. (TAR 2007) • National Association of Investors Corporation – 400,000 members • See Hodge (AH 2003)

  23. Source of Participants • Financial Analysts (sell and buy side) • Alumni Directory • Individual investment banks, financial management firms, pensions funds, etc. • Association for Investment Management and Research • Nelson’s Directory of Investment Managers

  24. Source of Participants • Standard setters and regulators • Good luck!

  25. Conclusions • Appropriate choice of participants requires a good research question, a good design and task, and careful consideration of how personal characteristics affect JDM • How does theory suggest different levels of a characteristic would change JDM?

  26. Conclusions • Be enthusiastic and able to explain why people should care about your research • Practice describing your research to your accountant and non-accountant friends • BE PERSISTENT !!! (but not annoying!)

  27. References • Berg, J.E., Daley, L.A., Dickhaut, J.W., and J. O’Brien. 1986. Controlling preferences for lotteries on units of experimental exchange. Quarterly Journal of Economics 101 (May): 281-306. • Elliott, W. B. 2006. Are investors influenced by pro forma emphasis and reconciliations in earnings announcements? The Accounting Review 81 (January): 113-133. • Elliott, W. B., F. Hodge, J. Kennedy, and M. Pronk. 2007. Are MBA students a good proxy for nonprofessional investors? The Accounting Review (January):139-168. • Hodge, F. D. 2003. Investors’ perceptions of earnings quality, auditor independence, and the usefulness of audited financial statements. Accounting Horizons 17(Supplement): 37-48. • Libby, R., Bloomfield, R., and M. Nelson. 2002. Experimental research in financial accounting. Accounting, Organizations and Society 27 (November): 775-810. • Libby, R. and J. Luft. 1993. Determinants of judgment performance in accounting settings: Ability, knowledge, motivation, and environment. Accounting Organizations and Society 18 (July):425-450. • McDaniel, L.S., Martin, R. D., and Maines, L.A. (2002). Evaluating financial reporting quality: The effects of financial expertise vs. financial literacy. The Accounting Review 77 (Supplement): 139-167. • Peecher, M. and I. Solomon. (2001). Theory and experimentation in studies of audit judgments and decisions: Avoiding common research traps. International Journal of Auditing (November): 193-203.

More Related