U.S. Army Research, Development and Engineering Command

1. U.S. Army Research, Development and Engineering Command Use of Evidence-based Strategies to Enhance the Extensibility of Adaptive Tutoring Technologies Benjamin Goldberg, Ron Tarr, Dr. Deborah Billings, Naomi Malone, Keith Brawner, and Dr. Robert Sottilare

2. Push for Tailored Training Computer-based tutoring systems (CBTS) have demonstrated significant promise in tutoring individuals in well-defined domains, but… Fifty years of research have been unsuccessful in making CBTS ubiquitous in military training… Why? CBTS are expensive to author and are insufficiently adaptableto support the tailored, self-regulated, individual & small unit tutoring experiences required to support: • U.S. Army Learning Model (ALM) for 2015 (TRADOC, 2011) • U.S. Air Force (AETC, 2008) • U.S. Navy STEM Grand Challenge (ONR, 2012) • OSD R&T Vision for PAL • NATO HFM RTG 237 (Advanced ITS) • TTCP HUM TP-2 (Training Panel)

3. Computer-Based Tutoring Systems (CBTS) • ITSs apply Artificial Intelligence tools and methods to individualize instruction • Based on benefits associated with one-on-one expert tutoring (2-Sigma Problem; Bloom, 1984) • Mediates learning by providing feedback when appropriate and adjusting difficulty levels to maintain desired challenge • Facilitated by 4 common components

4. Generalized Intelligent Framework for Tutoring (GIFT)

5. Pedagogical Modeling • Designed to balance the level of guidance a learner needs with the goal of maintaining engagement and motivation

6. Macro-Adaptive Strategies • Organized and sequential set of tactics to be implemented online • What to adapt and how to adapt • Addresses four instructional design areas: • Selection • Sequencing • Synthesizing • Summarizing • Based on metrics collected prior to the commencement of instruction

7. Macro-Adaptation:Sources of Adaptation Student Performance/ Achievement Levels Working Memory Capacity (WMC) “Sources of adaptation” refer to the factors that prompt, or trigger, adaptation to occur, namely the characteristics of learners that elicit specific instructional tactics to be implemented. Knowledge Type MATERIAL: Task Difficulty/ Complexity Source of Adaptation (Individual Differences & Task Characteristics) Learner’s Prior Knowledge/ Expertise Learner’s Traits/Ability/ Attributes Learning Styles/Cognitive Styles IMI Levels Task Learning Category: Cognitive, Affective, Psychomotor, Social

8. Macro-AdaptationTargets of Adaptation Different sources of adaptation appear to be linked with different targets of adaptation (as shown) in the literature. Based on: - The instructional elements that are adapted -Elements are adapted according to the specific sources of adaptation

9. Home Tutorial Instructor Training Developer The Instructional Strategies Indicator (ISI) is a computer-based searchable tool for selecting individualized instructional strategies based on the type of knowledge being learned, the domain in which the learning occurs, the expertise level of the learner, and the size of the group being taught. Our goal is to inform the selection of instructional strategies in order to improve learning effectiveness and efficiency across a wide range of domains. Ultimately, the ISI is intended to provide recommendations regarding how, what, and when to embed instructional components into simulation-based training systems. To start a search, click on the Strategies tab at the top. Log in: User name ______________ Password _________________

10. Application in GIFT:Model Development

11. Illustrative Example PRE-TRAINING [Human] Instructor Input PRE-TRAINING Assessment of Trainee Instructional Content Trainee Characteristics Declarative Novice Conceptual Journeyman Goal Orientation/Motivation Self-Efficacy Expertise Level Learning Objectives Knowledge Type Procedural Expert Integrative GIFT’s Pedagogical Model Higher Level Instructional Tactic Instructor Input Higher Level Instructional Tactic Increase “fog” in Scenario EXAMPLE: Increase Scenario Complexity

12. Extensibility • The need for Empirical Evaluation of a Generalized Pedagogical Model to determine its utility across multiple domains and training platforms • Goal of Experimentation • Determine best practices for “Best-in-Class” model to be incorporated in GIFT • Determine influence Individual Differences have on training outcomes • Determine how instructors and trainers will use prescribed outputs from model implementation

13. Road-Ahead • Development of a complementary Micro-Adaptive Pedagogical Model for use in GIFT • Used to inform ‘in-situ’ strategies and tactics on a general level (e.g., provide hint, adjust difficulty, perform assessment, etc.) • Requires techniques to monitor reactive states (i.e., cognitive and affective) and strategies to mitigate negative states • Soldier Centered Army Learning Environment (SCALE) • Utilize GIFT as SCALE’s pedagogical management engine • Provide mechanisms of personalized/tailored instruction with distributable training content

14. QUESTIONS