Agenda

1. I/ITSEC 2008 Tutorial:Virtual Training Simulations & Game-Based Systems: Large-Scale Adoption IssuesAmela Sadagic, PhDNaval Postgraduate School, MOVES Institute, asadagic@nps.edu

2. Agenda • Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

3. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

4. Tutorial is Not About… • Connectivity to the network & server needs • Costs & financial aspects • Acquisition & procurement • Maintenance & administrative management

5. Tutorial is About… • The phase AFTER the training system is acquired. • User aspects: dissemination, adoption, deployment and use of those systems among intended users (both instructors and trainees). • Factors influencing large scale adoption. • Techniques for changing adoption rate. • Effective techniques for maximizing the investments made in those systems.

6. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

7. Current Training Needs • Situationthat military community deals with: • changes of doctrinal teachings & mission objectives, • operational tempo changes dramatically, • unsatisfactory retention rate for the serviceman • … and no performance drop-off! • Training needs: • train large number of skills, • train large number of people, • train in novel ways, motivate learners, • train in novel places & under novel conditions, • acquire new skills, • learn & do novel tasks, • … and achieve all that in a short period of time!

8. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

9. Simulations and Game-Based Systems • Can they be a solution? • They will not provide a complete (only) solution, • … but most likely they will be a good part of that solution. • A paradigm shift - truly and successfully enabling novel training practices, and achieving significant results, happens only when: • large majority (ideally everyone) uses those solutions, • they do it methodically and consistently, • they have those solutions available 24 / 7.

10. Definition of Problem • Affordable solutions, substantial and continuing investments, fairly well recognized and acknowledged potential… • … yet still no evidence of large scale adoption of technology-based solutions and their effective and systematic use for learning and training purposes. • Large scale: > 80 % users

11. Simulations and Game-Based Systems …cont. Game-based systems: computer-supported real-time systems that couple multiple sensory information (visual, auditory, haptic, olfactory) in an organized way providing a meaningful context for human action and collaboration. This includes the elements of: • content: representation of the environment, actors and characters (one or many), • storyline / plot / scenario, • task(s), • dynamics: set of rules, behaviors and interaction modalities.

12. Examples of Games and Game-Based Systems Games -> Entertainment: • Quake • Warcraft Reloaded • Tiger Woods 99 PGA Tour Golf Game-Based Systems / Serious games -> Learning and Training: • CCM (Close Combat Marine) • VBS 2 (Virtual Battle Space) • FOPCSim, VCCT Which group is concerned with the validity, correct simulation of physical phenomena and human behaviors, metrics and measurements?

13. Game-Based Training Systems: Examples

14. Simulations and Game-Based Training Systems Why should we use them? They are: • mature enough, • affordable, • they have a potential to: • Enable more effective learning/training: • learn more, quicker, retain skills and knowledge longer, less cost involved, • Increase interest and motivate learners/trainees, • Enable learning/training situations that would not be possible otherwise, • But also…

15. …Correspond to Contemporary Lives of Its Users • 3 years ago: > 60% Marines played video games. They also used other digital gadgets & applications. • Now: All young IOC and TBS officers, as well as young Marines (almost everyone) own personal computer. Questions: • Should learners’ working hours (school, unit) be as contemporary as their free-time? • Is the alienation from a ‘clunky’ & ‘old’ segment of learner’s life possible to happen, and how can one address it?

16. vs. Classroom-of-the-Future(as seen by the high-school students 8 years ago in 2000!) Expectations set by new generations:

17. More on ‘Why Use Game-Based Training Systems…’ • Provide rich visual and spatial representations • Simulate rich environment with multiple sensory information coupled together in an organized way - video, audio and other stimuli in a sync • Provide immediate feedback to learner’s actions • Include the elements of storytelling and narration • Enable role-playing and experimentation • Engage user in active learning process • Can be fun and motivate users for learning (training) • Adaptable for different skill levels and learning styles • Enable high level of presence (this may influence performance) • Immerse users in problem-solving activities • Enhance experiential learning • Easy to play out a number of different situations (scenarios) - perfecting skills • Exploring a number of what-ifs • Self-selection of the level of difficulty – ownership over the learning process

18. Caveats • Simulations (technology) are only the tools - not a goal and not a ‘full’ package, just a segment of it • Simulations are not the ultimate replacement for current training approaches. • Simulation should be employed when it is a better solution for a given objective - need to match training approaches and tools with training objectives. • Efforts should be directed towards coupling of learning/ training objectives and goals with right approaches, right tools, having in mind the audience we deal with.

19. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

20. Diffusion of Innovation • It is a technical issue AND social process. • Innovations do NOT sell themselves. 5 categories of adopters: • Innovators • Early adopters • Early majority • Late majority • Laggards • Opinion leaders - change agents - change agent aide Everett M. Rogers, Diffusion of Innovation

21. Diffusion of Innovation …cont. • time Cummulative diffusion Everett M. Rogers, Diffusion of Innovation

22. Diffusion of Innovation …cont. • Mass-media channels vs interpersonal channels. • Majority: form their opinion on the basis of subjectiveevaluation of information received from their peers (similar socioeconomic status education and other values). • Interpersonal channels: most effective for ‘majority’ adopters. • Fastest adoption: decision coming from the authority. • An innovation should not be considered in isolation from other innovations. Everett M. Rogers, Diffusion of Innovation

23. Factors Influencing Adoption Rate • Relative advantage: benefits over current solution. • Compatibility: degree of being consistent with current system of values. • Complexity: simpler to understand & simpler to use. • Trialability: adoption in an incremental fashion. • Observability: results being visible to other adopters. Everett M. Rogers, Diffusion of Innovation

24. Relative Advantage: Issues • (definition: perceived benefits over current solution): • Absence of ‘full package’ solutions • Black-box solution… just a wishful thinking • Lack of formal training for people who deliver instruction • One time exposure only + short exposure • Timing not appropriate • Wrong order of skill mastery • Disconnect between the systems and (right) users • No syllabus, no high-quality scenarios • Not matching system capabilities and levels with users • Training or ‘fun’? • Missing training relevance (need for continuous updates) • Lack of evaluation of training effectiveness • Lack of accountability for achieved training results • Motivation: “Checking the ‘technology’ box”

25. Compatibility: Issues • (definition: degree of being consistent with current system of values) • Lack of system support for After Action Review (AAR) Record your session, appoint the ‘observers” & ‘evaluators’ • Do I need to forget everything I knew and did so far? Explore a synergy of old (known, tested) technologies/ methods and simulations - that mix may well be the best solution for given training objective!

26. Complexity: Issues • (definition: simpler to understand & simpler to use -> adopted more rapidly) • Do I need to be a technology expert? • One possible solution: trainees acting as (occasional) technical support. • User interfaces, user navigation and interaction modalities, and key combinations differ from one system to another: • Learning new system is more difficult - making those characteristics more uniform across the systems would help

27. Trialability: Issues • (definition: adoption in an incremental fashion -> adopted more rapidly) • Rigid definition of what it means to ‘use the technology’: 100%, 50% or 3% of training time? • Should all trainees use computers… or not?  Consider different combinations and arrangements with some people using the system and some not using it.

28. Observability: Issues • (definition: results being visible to other adopters -> adopted more rapidly) • Mandatory use of simulations (e.g. aviation; ship, submarine & tank navigation; missile engagements): • Results were substantial, tangible, clearly visible, with high relative advantage and immediate • Optional use of simulations (e.g. tactical decision-making skills): • Relative advantage visible only after a long term use  Advertise the successes of peers in their community

29. Influencing Users’ Attitude • “This will help us reduce the number of instructors” - hope for BIG savings • “This system is all you will ever need” • “These systems will (should) sell themselves”, • “It will be a ‘pull’ process only, no need for ‘push’ strategies”. • “We will start preparing for deployment once the training system is acquired.” • Unrealistic expectations on learning results and timing • more complex & more expensive solution -> higher the expectations • time to get acquainted with the system - it is a process • Accountability for the skill transfer (field performance)

30. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

31. Examples of (Partial) Success • America’s Army • It is a game. Started as a recruitment tool, not as training system. • Substantial promotional efforts invested, and they are still present • Has active support: support forums, organized events for peers (gamers) and chat networks, • Professional web-site with • expected segments focused on • engaging new players: expanding • user base is their mission. • No user (‘transfer of training’) • studies done. 31 (image taken from America’s Army web site)

32. Examples of (Partial) Success • FOPCSim (Forward Observer PC Simulation): • Training system designed by Marines (MOVES students) • Actively promoted by Marines - available in each Simulation center • Agent of change was one of original designers. He is also artillery officer i.e. peer of • targeted end users. • Includes good & tested scenarios • Uses Delta3D game engine – no • license fees involved. • User studies proved its • training effectiveness 32

33. Examples of (Partial) Success • VBS 2 (Virtual Battle Space): • Training system • Actively promoted by Marines - Available in each Simulation center • Continually perfected and fine-tuned to fit training needs • Company offers courses for administrators and developers & • support for users (ex-military • contractor team  experienced • as very close to peers/users) • Supports large spectrum of • training situations. 33 (image taken from VBS 2 web site)

34. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

35. Practical Considerations • Q1: Do my trainees need any prep time before they • engage in active learning/training? • Q2: Do I need to bring any aids? Cards? Projector? • Recording devices? And how about the use of • headphones? • Q3: What should I do when the trainees start gaming • (playing around, using non-doctrinal tactics)? • Q4: How should I arrange my computers (I need to • organize a session for an entire group)? • Q5: Should I think about introducing elements of a • challenge or competition?

36. Learners as Your Technical Support Members? • Q:Is it realistic to expect that all instructors will have necessary technical expertise and experience? • Trainees acting as (occasional) technical support: • active involvement vs. ‘being served’ approach • great opportunity to learn more about technology - they may need those skills in very near future, • recognition of their skills, • instills a sense of ownership over the process, • higher appreciation for instructor’s efforts, • more forgiving when technical difficulties are experienced.

37. Everyone Using the System… or Not? Q (Valid concern): Certain tasks represent considerable cognitive load for an individual - do I add to that by asking them to control an input device in addition to their already complex tasks? Q: Could the experience of watching someone use the system be another form of learning? • Consider different combinations and arrangements with some people using the system and some not using it. • The goal: ALL trainees should benefit from that arrangement and that session.

38. Combinations With Other (Older) Instructional Approaches • Q: How beneficial is it to use a combination of new and old tested instructional approaches? • What is the potential they may bring? • They work (quite often very successfully). • This combination may be the best fit for training (match with beginning, intermediate, advanced levels). • ‘Old’ approaches serving as ‘suspenders’ in case of hasty behaviors while training with simulations. • Instructors are familiar with them - may be more inclined to accept a combination then simulation-only approach.

39. Combinations With Other Technologies and Media • Q: How beneficial is it to use a combination of different technologies and media? • Consider: Synchronous & asynchronous tools Before, during & after the session • Options: Chat (audio and text) Blog as a project diary (text blog, video blog, podcasts) Videoconferencing

40. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

41. Increasing Adoption Rate • time Cummulative diffusion

42. Increasing Adoption Rate • Knowing the characteristics of military as a social system: • Introduction of mandatory deployment and use of simulations • If you do, make sure there is a strong and valid rationale for such decision. Also, make sure it is accepted on ALL levels. • Increase the number of agents of change: • Create new billets dedicated to dissemination and use of simulations + make simulation focus be the main focus • More active and changed role for simulation centers • Introduce challenge programs and competitions.

43. Domain: What is Covered in This Tutorial • Current Training Needs • Simulations and Game-Based Systems • Diffusion of Innovation • Examples of (Partial) Success • Practical Considerations and Techniques • Increasing Adoption Rate • Conclusion and Q&A

44. References • Malcolm Gladwell (2000), The Tipping Point, Little, Brown and Company, 2000. • Joseph M. Nolan and Jason M. Jones (2005), Games For Training: Leveraging Commercial Off The Shelf Multiplayer Gaming Software For Infantry Squad Collective Training, Master Thesis , NPS. • Baxter, Holly C., Ross, Karol G., Phillips, Jennifer, Shafer, Jennifer, Fowlkes, Jennifer. (2004). Leveraging Commercial Video Game Technology to Improve Military Decision Skills. Inter-service/Industry Training, Simulation, and Education Conference (I/ITSEC) 2004. • Prensky, M. (2001). Digital Game Based Training. New York: McGraw-Hill, 2001. • Everett M. Rogers (1995), Diffusion of Innovation, The Free Press 1995. • Amela Sadagic (2007), The Deployment and Use of Virtual Training Simulations: What Does it Take to Serve the Needs of Majority Of Its Users?, New Learning Technologies Orlando 2007 SALT Conference, Orlando, FL, Jan 31 - Feb 2, 2007.

45. References …cont. • Amela Sadagic, Rudy Darken (2006), Combined Arms Training: Methods and Measures for a Changing World, NATO workshop Virtual Media for Military Applications, US Military Academy, West Point, NY, 13-15 June 2006. • Robert Sibley and Amela Sadagic (2003), Emerging Technologies as Enablers of Advanced Teaching and Learning Practice, National Educational Computing Conference - NECC 2003, Seattle WA, July 2003. • Wayne Zachary, Robert R. Hoffman, Kelly Neville, Jennifer Fowlkes (2007), Human Total Cost of Ownership: The Penny Foolish Principle at Work, IEEE Intelligent Systems, March/April 2007.

46. Q & A