Virtual Training Simulations & Game-Based Systems: Large-Scale Adoption Issues

SpringSim, Military Modeling & Simulation, San Diego, CA, March 2009 (tutorial also presented at I/ITSEC 2008) Virtual Training Simulations & Game-Based Systems: Large-Scale Adoption Issues Amela Sadagic, PhD asadagic@nps.edu

Motivation Late 80s, early & mid 90s

Motivation

Agenda • Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

Tutorial is Not About… • Connectivity to the network & server needs • Costs & financial aspects • Acquisition & procurement • Maintenance & administrative management

Tutorial is About… • The phase AFTER the training system is acquired. • User aspects: dissemination, adoption, deployment and use of those systems among intended users (both instructors and trainees). • Factors influencing large scale adoption. • Techniques for changing adoption rate. • Effective techniques for maximizing the investments made in those systems.

Current Training Needs Situation that military community deals with: • changes of doctrinal teachings & mission objectives, • operational tempo changes dramatically, • unsatisfactory retention rate for the serviceman, • … and no performance drop-off! Training needs: • train large number of skills, • train large number of people, • train in novel ways, motivate learners, • train in novel places & under novel conditions, • acquire new skills, • learn & do novel tasks, • … and achieve all that in a short period of time!

Simulations and Game-Based Systems Can they be a solution? • They will not provide a complete (only) solution, • … but most likely they will be a good part of that solution. A paradigm shift - truly and successfully enabling novel training practices, and achieving significant results, happens only when: • large majority (ideally everyone) uses those solutions, • they do it methodically and consistently, • they have those solutions available 24 / 7.

Definition of Problem Affordable solutions, substantial and continuing investments, fairly well recognized and acknowledged potential… … yet still no evidence of large scale adoption of technology-based solutions and their effective and systematic use for learning and training purposes. Large scale: > 80 % users

Simulations and Game-Based Systems …cont. Game-based systems: computer-supported real-time systems that couple multiple sensory information (visual, auditory, haptic, olfactory) in an organized way providing a meaningful context for human action and collaboration. This includes the elements of: • content: representation of the environment, actors and characters (one or many), • storyline / plot / scenario, • dynamics: set of rules, behaviors and interaction modalities, • task(s) and overall goal of action, • system feedback about player’s success/results in session (score). Goal: learning and training.

Examples of Games and Game-Based Systems Games -> Entertainment: • EverQuest • World of Warcraft • Tiger Woods 99 PGA Tour Golf Game-Based Systems / Serious Games -> Learning and Training: • CCM (Close Combat Marine) • VBS 2 (Virtual Battle Space) • FOPCSim, VCCT Which group is concerned with the validity, correct simulation of physical phenomena and human behaviors, metrics and measurements?

Game-Based Training Systems: Examples

Simulations and Game-Based Training Systems Why should we use them? They are: • mature enough, • affordable, • they have a potential to: • Enable more effective learning/training: • learn more, quicker, retain skills and knowledge longer, less cost involved, • Increase interest and motivate learners/trainees, • Enable learning/training situations that would not be possible otherwise, • But also…

…Correspond to Contemporary Lives of Its Users • 4 years ago: > 60% Marines played video games. They also used other digital gadgets & applications. • Now: All young IOC and TBS officers, as well as young Marines (almost everyone) own personal computer. Questions: • Should learners’ working hours (school, unit) be as contemporary as their free-time? • Is the alienation from a ‘clunky’ & ‘old’ segment of learner’s life possible to happen, and how can one address it?

vs. Classroom-of-the-Future (as seen by the high-school students 9 years ago in 2000!) Expectations set by new generations:

More on ‘Why Use Game-Based Training Systems…’ • Provide rich visual and spatial representations • Simulate rich environment with multiple sensory information coupled together in an organized way - video, audio and other stimuli in a sync • Provide immediate feedback to learner’s actions • Include elements of storytelling and narration • Enable role-playing and experimentation • Engage user in active learning process • Can be fun and motivate users for learning (training) • Adaptable for different skill levels and learning styles • Enable high level of presence (this may influence performance) • Immerse users in problem-solving activities • Enhance experiential learning • Easy to play out a number of different situations (scenarios) - perfecting skills • Exploring a number of what-ifs • Self-selection of the level of difficulty – ownership over the learning process

Caveats • Simulations (technology) are only the tools - not a goal and not a ‘full’ package, just a segment of it. • Simulations are not the ultimate replacement for current training approaches. • Simulation should be employed when it is a better solution for a given objective - need to match training approaches and tools with training objectives. • Efforts should be directed towards coupling of learning/ training objectives and goals with right approaches, right tools, having in mind the audience we deal with.

Diffusion of Innovation • It is a technical issue AND social process. • Innovations DO NOT sell themselves. 5 categories of adopters: • Innovators • Early adopters • Early majority • Late majority • Laggards Opinion leaders - change agents - change agent aide Everett M. Rogers, Diffusion of Innovations

Diffusion of Innovation …cont. time Cummulative diffusion Everett M. Rogers, Diffusion of Innovations

Diffusion of Innovation …cont. • Mass-media channels vs interpersonal channels. • Majority: form their opinion on the basis of subjectiveevaluation of information received from their peers (similar socioeconomic status, education and other values). • Interpersonal channels: most effective for ‘majority’ adopters. • Fastest adoption: decision coming from the authority. • An innovation should not be considered in isolation from other innovations. Everett M. Rogers, Diffusion of Innovations

Factors Influencing Adoption Rate • Relative advantage: benefits over current solution. • Compatibility: degree of being consistent with current system of values. • Complexity: simpler to understand & simpler to use. • Trialability: adoption in an incremental fashion. • Observability: results being visible to other adopters. Everett M. Rogers, Diffusion of Innovations

Relative Advantage: Issues (definition: perceived benefits over current solution): • Absence of ‘full package’ solutions • Black-box solution… just a wishful thinking • Lack of formal training for people who deliver instruction • One time exposure only + short exposure • Timing not appropriate • Wrong order of skill mastery • Disconnect between the systems and (right) users • No syllabus, no high-quality scenarios • Not matching system capabilities and levels with users • Training or ‘fun’? • Missing training relevance (need for continuous updates) • Lack of evaluation of training effectiveness • Lack of accountability for achieved training results • Motivation: “Checking the ‘technology’ box”

Compatibility: Issues (definition: degree of being consistent with current system of values) • Lack of system support for After Action Review (AAR) Record your session, appoint the ‘observers” & ‘evaluators’ • Do I need to forget everything I knew and did so far? Explore a synergy of old (known, tested) technologies/ methods and simulations - that mix may well be the best solution for given training objective!

Complexity: Issues (definition: simpler to understand & simpler to use  adopted more rapidly) • Do I need to be a technology expert? • One possible solution: trainees acting as (occasional) technical support. • User interfaces, user navigation and interaction modalities differ from one system to another: • Learning new system is more difficult - making those parameters uniform/same across different systems would help.

Trialability: Issues (definition: adoption in an incremental fashion  adopted more rapidly) • Rigid definition of what it means to ‘use the technology’: 100%, 50% or 3% of training time? • Should all trainees use computers… or not?  Consider different combinations and arrangements with some people using the system and some not using it.

Observability: Issues (definition: results being visible to other adopters -> adopted more rapidly) • Mandatory use of simulations (e.g. aviation; ship, submarine & tank navigation; missile engagements): Results were substantial, tangible, clearly visible, with high relative advantage and immediate • Optional use of simulations (e.g. tactical decision-making skills): Relative advantage visible only after a long term use  Advertise the successes of peers in their community.

Influencing Users’ Attitude • “This will help us reduce the number of instructors” - hope for BIG savings. • “This system is all you will ever need.” • “These systems will (should) sell themselves.” • “It will be a ‘pull’ process only, no need for ‘push’ strategies.” • “We will start preparing for deployment once the training system is acquired.” • Unrealistic expectations on learning results and timing • More complex & more expensive solution  higher the expectations. • Time to get acquainted with the system - it is a process. • Accountability for the skill transfer (field performance).

Examples of (Partial) Success America’s Army • It is a game. Started as a recruitment tool, not as a training system. • Substantial promotional efforts invested, and they are still present. • Has active web support: support forums, organized events for peers (gamers) and chat networks. • Professional web-site with segments focused on engaging new players: expanding user base is their mission. • No user studies done (‘transfer of training’). (image taken from America’s Army web site)

Examples of (Partial) Success FOPCSim (Forward Observer PC Simulation) • Training system designed by Marines (MOVES students). • Actively promoted by Marines - available in each Simulation center. • Agent of change was one of original designers. He is also artillery officer i.e. peer of targeted end users. • Includes good & tested scenarios. • Uses Delta3D game engine – no license fees involved. • User studies proved training effectiveness.

Examples of (Partial) Success VBS 2 (Virtual Battlespace 2) • Training system. • Actively promoted by Marines - Available in each Simulation Center. • Continually perfected and fine-tuned to fit training needs. • Company offers courses for administrators and developers & support for users (ex-military contractor team  experienced as very close to peers/users) • Supports large spectrum of training situations. • Still need to perfect their simulations. (image taken from VBS 2 web site)

Practical Considerations Q1: Do my trainees need any prep time before they engage in active learning/training? Q2: Do I need to bring any learning/training aids? Cards? Projector? Recording devices? And how about the use of headphones? Q3: What should I do when the trainees start gaming (playing around, using non-doctrinal tactics)? Q4: How should I arrange my computers (I need to organize a session for an entire group)? Q5: Should I think about introducing elements of a challenge or competition?

Learners as Your Technical Support Members? Q: Is it realistic to expect that all instructors will have necessary technical expertise and experience? A: Trainees acting as (occasional) technical support: • active involvement vs. ‘being served’ approach, • great opportunity to learn more about technology - they may need those skills in very near future, • recognition of their skills, • instills a sense of ownership over the process, • higher appreciation for instructor’s efforts, • more forgiving when technical difficulties are experienced.

Everyone Using the System… or Not? Q (Valid concern): Certain tasks represent considerable cognitive load for an individual - do I add to that by asking them to control an input device in addition to their already complex tasks? Q: Could the experience of watching someone use the system be another form of learning? A: Consider different combinations and arrangements with some people using the system and some not using it. The goal: ALL trainees should benefit from that arrangement and that session.

Combinations With Other Instructional Approaches Q: How beneficial is it to use a combination of new and old tested instructional approaches? A: Potential benefits: • They work (quite often very successfully). • This combination may be the best fit for training (match with beginning, intermediate, advanced levels). • ‘Old’ approaches serving as ‘suspenders’ in case of hasty behaviors while training with simulations. • Instructors are familiar with them - may be more inclined to accept a combination then simulation-only approach.

Combinations With Other Technologies and Media Q: How beneficial is it to use a combination of different technologies and media? Consider: Synchronous & asynchronous tools Before, during & after the session Optional (additional) media: Chat (audio and text) Blog as a project diary (text blog, video blog, podcasts) Social networking Videoconferencing

Increasing Adoption Rate time Cummulative diffusion

Increasing Adoption Rate Knowing the characteristics of military as a social system: • Introduction of mandatory deployment and use of simulations • If you do, make sure there is a strong and valid rationale for such decision. Also, make sure it is accepted on ALL levels. • Increase the number of agents of change: • Create new billets dedicated to dissemination and use of simulations + make simulation focus be the main focus • Make some training tools/systems ‘unit/Marine-centered’ instead of ‘simulation center-centered’ • More active and changed role for simulation centers. • Introduce challenge programs and competitions.

References • Malcolm Gladwell (2000), The Tipping Point, Little, Brown and Company, 2000. • Joseph M. Nolan and Jason M. Jones (2005), Games For Training: Leveraging Commercial Off The Shelf Multiplayer Gaming Software For Infantry Squad Collective Training, Master Thesis , NPS. • Baxter, Holly C., Ross, Karol G., Phillips, Jennifer, Shafer, Jennifer, Fowlkes, Jennifer. (2004). Leveraging Commercial Video Game Technology to Improve Military Decision Skills. Inter-service/Industry Training, Simulation, and Education Conference (I/ITSEC) 2004. • Prensky, M. (2001). Digital Game Based Training. New York: McGraw-Hill, 2001. • Everett M. Rogers (1995), Diffusion of Innovations, The Free Press 1995. • Amela Sadagic (2007), The Deployment and Use of Virtual Training Simulations: What Does it Take to Serve the Needs of Majority Of Its Users?, New Learning Technologies Orlando 2007 SALT Conference, Orlando, FL, Jan 31 - Feb 2, 2007.

References …cont. • Amela Sadagic, Rudy Darken (2006), Combined Arms Training: Methods and Measures for a Changing World, NATO workshop Virtual Media for Military Applications, US Military Academy, West Point, NY, 13-15 June 2006. • Robert Sibley and Amela Sadagic (2003), Emerging Technologies as Enablers of Advanced Teaching and Learning Practice, National Educational Computing Conference - NECC 2003, Seattle WA, July 2003. • Wayne Zachary, Robert R. Hoffman, Kelly Neville, Jennifer Fowlkes (2007), Human Total Cost of Ownership: The Penny Foolish Principle at Work, IEEE Intelligent Systems, March/April 2007.

Q & A

Virtual Training Simulations & Game-Based Systems: Large-Scale Adoption Issues