Factors Affecting DOD FM Usage of a GDSS

1. Factors Affecting DOD FM Usage of a GDSS Jeff Bohler & Dianne Hall, PhD Auburn University Pre-ICIS 2006 SIGDSS Research Workshop 10 December, 2006 The opinions and observations expressed in this presentation are mine alone and do not represent the opinions of the DOD, the USAF, or the USN.

2. Overview • Research Motivation & Relevance • Decision Support Method • GRASP • Research Model & Hypotheses • Methodology & Analysis • Results • Limitations & Future Research

3. Research Motivation & Relevance • Changes in the DOD workforce • Personnel reductions • Retirement of “Baby Boom” generation • Shortage of analytical skills in FM • Financial Manager & Comptrollers School • Decision Support concept introduced • What makes this study interesting? • Decision Support training effectiveness • Professional population • Many IT, GSS, DSS changes underway in DOD

4. GRASP - BackgroundGroup Resource Allocation Simulation Program • Course includes “Strategic Economics” • Macro view of political, military, and economic relationships • GRASP Usage • Capstone exercise, used in both curricula • Decision Task • National level budget decisions • Group oriented (president & cabinet) • Face to face, synchronous, no “right” answer • Multiple objectives to optimize

5. GRASP – As a GDSSGroup Resource Allocation Simulation Program • Contains a body of knowledge (DIKW) • Data + Rules + Relationships + Context = Knowledge • Record keeping ability • Displays data history (17 variables / 4 turns) • Provides standardized reports to the decision maker • Allows decision maker choice in KM activities • Enables decision support from “cabinet” • Provides estimates of likely outcomes from decisions => new knowledge

6. Relevant Literature • Carroll & Johnson (1990) • Temporal stages of decision making • Nunamaker, Dennis, et al. (1991) • GSS Research Framework • Zopounidis et al. (1997) • Survey of DSS in financial management • Proposed a “Knowledge Based DSS” • Hall & Paradice (2003) • Adapted Simon’s IDC model, incorporating KM

7. GRASP Decision Support Method • Identify Issue • Analyze Issue • Develop Alternatives • Evaluate Alternatives • Make a recommendation • DS provides a framework for FM&C course • Addresses need for DS training in DOD FM • Congressional call for improved FM practice

8. Research Hypotheses H1: Groups that only use FtF communication methods will achieve greater decision quality than groups that use hybrid proximity (FtF & paperless). H2: Groups with higher complexity tasks (wars) will achieve lower decision quality than groups that have lower complexity tasks. H3: Groups that display greater adaptation of GRASP capabilities (intelligence gathering) will achieve greater decision quality than groups with a lower rate of adoption. H4: Groups that used DS methods will achieve greater decision quality than groups that did not.

9. Analysis Model Communication H1 H2 Complexity Outcome (DQ) H3 Adaptation H4 Goal Accomplishment DS Method Used

10. Methodology • Subjects (~850) randomly assigned Seminars balanced for military/civilian/job • Each seminar was split into two teams: • President is self selected • President picks cabinet (4 – 6 members) • Resulted in n=172 teams • Competition • A faculty facilitated exercise = lab experiment • Four years of exercise data • Used for debriefing, student feedback

11. Analysis • Linear Regression Model: AR2 = .291 • Decision Quality = Communication + Complexity + Adaptation + DS Model Use • Operationalizing Decision Quality… • Analysis of variance (ANOVA): • Unequal cell sizes • Brown-Forsythe test (constancy of error variance)

12. Results (at α= 0.05) • H1: Communication – Not supported • Beta = -0.097, p = 0.228 • H2: Complexity – Supported • Beta = -0.532, p = 0.000 • H3: Adaptation – Not supported • Beta = 0.540, p = 0.410 • H4: DS Model – Supported • Beta = 0.170, p = 0.034

13. Limitations & Future Research • Observational study • Hard to know cause & effect relationship • Other possible explanatory factors • More detailed analysis required • Content analysis of user reports will allow a better understanding of the decisions made, priorities, & tradeoffs. • Gather more data & use PLS • Future research • The decision making environment of DOD financial managers

14. Summary • Simulations useful, but limited: • Allows isolation of decision behaviors (individual, group, organization) • Limited generalizability to other groups • Need more research on decision support • Role of financial managers in decision making • Effectively decision support training methods • Questions?

15. GRASP Exercise Model Instruction Factors: - Instructor - Course type - Decision Support Concept - Decision Support Methodology Feedback - Internal - External Class Factors: - Class Timing - Class Size - Class Character - Year Attended 4th year Score Decision Support Group Decision Support Decision Maker Decision Quality Score Group Factors: - Group Size - Group Character Bonus Points Task Factors: - Complexity +Wars +Communications - Time Available - Resources Available Personal Factors: - Computer Literacy - Education Level - Military / Civilian - Rank - Gender Environmental Factors: - Competition Stress - External Distractions - Algorithm Anomalies

16. GSS Research Framework(Nunamaker, Dennis et al., 1991) adapted H1:Communication H4:DS Methods Used n ~ 850 Randomly Assigned Group H3:Adaptation H2:Complexity Intervening & Adaptation Process Task Outcome Controlled: • Instructor • Course • Phase Context DV: Decision Quality DSS Decision Group Environment Feedback Organizational Decision-Making Environment Next Phase / PLS GRASP