"Serious Communication for Serious Games" By Ross Kukulinski- Serious Play Conference 2012

1. Serious Communication for Serious Games Ross Kukulinski ross.kukulinski@asti-usa.com Serious Play Conference

2. Photo by Derek Jensen 2

3. Photo Courtesy of U.S. Military 3

4. Roadmap 1. Serious games and teamwork 2. Communications modeling 3. Fidelity 4. Game integration 5. Network interoperability 6. Final thoughts and future directions 4

5. Part 1 Serious Games are Powerful Tools 5

6. Serious Games … • Allow soldiers to experience situations that are impossible in the real world1 • Provide improved hand-eye coordination, multi-tasking, and teamwork2 • Are uniquely flexible to support varied training needs 1 Corti, 2006; Squire & Jenkins, 2003 2 Michael & Chen, 2006 6

7. ‘Good’ Serious Games Six Ingredients to a ‘good’ game1 1. Mechanics 2. Rules 3. Immersive Graphics 4. Interactivity 5. Challenge 6. Risks 7. What about communication? 1 Derryberry, 2007 7

8. Fundamentals of Teamwork The Big Five Core Components of Teamwork1 1. Team Leadership 2. Performance Monitoring 3. Backup Behavior 4. Adaptability 5. Team/Collective Orientation Hypothesis: Communication key element? 1 Salas, Sims, & Burke, 2004 8

9. Communication and Performance • America’s Army experiments – Researchers measured team communication • Communication network level • Number of report-ins • Number of normal communications • Teams with regular organized reports had: – Higher performance – Higher estimated situational awareness Schneider & Carley, 2005 9

10. DARWARS Ambush!: Authoring Lessons Learned in a Training Game1 • Communication skills are critical for success – Requires effective communications training • Communications capabilities differ widely across varying military units • Training system should be similar to real- world communication system 1 Diller, Orberts, Blankenship, Nielsen, 2004 10

11. DARWARS Lessons Cont’d • Primary functions of a convoy commander – Establish and maintain communications within the convoy – Maintain communication with superordinate and subordinate element commanders Diller, Orberts, Blankenship, Nielsen, 2004 11

12. A Training Transfer Study of Serious Games “Our work in this project demonstrated consistently through all five experiments that communications is fundamental to the training experience and one of the most important aspects of the exercise.” Major Ben Brown MOVES Institute Naval Postgraduate School Brown, 2010 12

13. Game Communication Options • Nothing • Text-chat • Game integrated voice communication • Third-party voice communication None simulate real-world communication! 13

14. Our Customer Feedback • Some were content with what they had • Some engineered custom solutions • Many were frustrated – Current game communication systems… • are not robust • are difficult to manage in large installations • do not simulate real-world radio communication • do not integrate well with other training systems • lack live technical support and expertise 14

15. Brief History Lesson • Full Spectrum Warrior – 2000-2003 • America’s Army – 2002 • DARWARS Ambush – 2005-2009 • ‘Game After Ambush’ (VBS2) – 2009-2012/13 • ‘Games For Training’ Recompete – Q4-2012 15

16. Communication Specifications • Game After Ambush (2009) – 8,118 words in technical specification – 128 words for describing communication • Games for Training (Draft – October 2011) – 5,113 words in technical specification – 128 words for describing communication • Games for Training (Draft – April 2012) – Finally requires high-fidelity voice communication 16

17. Part 2 Communications Modeling 17

18. Modes of Voice Communication • Intercoms • Radios • Earshot 18

19. Intercoms • Full-duplex • One channel per ‘wire’ • 1-to-1 or n-to-n participants • Phone/Conference call Photo by K!T 19

20. Radios • Half-duplex (usually) • 1-to-n participants • Many configurations possible (AM, FM, PT/CT, freq, encoding, etc.) • Variable communication link quality • Complex and hard to simulate in real-time • Noisy! Photo courtesy of U.S. Military 20

21. Earshot • Simulated voice communication – Full duplex – Volume and quality degrades over distance 21

22. Simulating Real-world Communication • Simulated radios behave like real-world radios – AM, FM, Frequency Hopping – Half-duplex radios – Full-duplex intercoms – Real-time dynamic radio noise – Realistic propagation effects due to ranging, occulting, radio power level, and terrain – Crypto system sound effects 22

23. Simulating Real-world Communication • Simulate voice communication (Earshot) – Volume and quality degrades over distance – Separate from radio simulation • Trainees limited to channels they would have in real-world 23

24. Part 3 Fidelity 24

25. Acceptable Fidelity • What is the training goal? • What is the real-world communication? • Combined arms, convoy, and small unit communications “must be correct and effective” • “One can debate the level of fidelity needed for useful training, but fidelity must certainly be high when it relates to the specific task being trained” 1Brown, 2010 25

26. Basic Intercom 26

27. Intercom and Individual Radios 27

28. Geolocated Individual Radios 28

29. Geolocated Vehicle & Individual Radios 29

30. Highest Fidelity 30

31. Part 4 Game Integration 31

32. “Quite simply, communications should be as seamless as all other aspects of [the serious game]. Communications should be internal to [the game] with seams between vendor production transparent to the user.”1 1Brown, 2010 32

33. Administrative Interface • In addition to personnel and weapons, game scenarios should include communication • Radio/intercom configuration and allocation • DIS/HLA configuration • Re-usable communication configuration 33

34. User Interface • Two possible views – Heads Up Display (HUD) – In-game objects • HUD – Simple and intuitive – Not realistic – does it break flow? • In-game objects – Higher realism – does it impede training? 34

35. User Interface Capabilities • Regardless of view mode: – Support for multiple radios and intercoms – View radio channel and Tx/Rx status – Support changing radio channels – Dynamic vehicle communication systems – Earshot voice communication 35

36. Required Simulation Information • Radio location from game entities for realistic radio effects like ranging and occulting • Player location for Earshot voice communication • Assign radios and intercoms to vehicles for mounted training – Players acquire vehicle-based radios when mounted, lose access when dismounted 36

37. After Action Review • “Both simulation groups commented extensively on the AAR tool. Both groups believed the AAR tool was critical in providing a big picture view of what happened during the exercise.” 1 • Communication playback synced with visuals • Seek, Pause, FF, RW, Bookmarks • Export audio/visual for later analysis and study 1Brown, 2010 37

38. Part 5 Network Interoperability 38

39. Live-Virtual-Constructive Example 39

40. DIS & HLA • Distributed Interactive Simulation – Wire-level specification – UDP – Simple! – But standard slow to evolve • High-Level Architecture – Run Time Infrastructure – Set of API functionality – Federation Object Model – Very complex 40

41. Networked Voice • Audio encoding and sample rate • Dynamic packet sizes • Latency – Maximum 150ms one-way latency – Latency <100 ideal • Unreliable networks (jitter, lost packets) 41

42. Part 6 Final Thoughts 42

43. Summary • Communication is critical for teamwork • Real-time communication simulation is computationally complex • Serious games require high-fidelity communication for effective training • Requirement to network disparate training systems into a common network 43

44. Resources • • Brown, B., (2010) A Training Transfer Study of Simulation Games Carpenter, R., White, C., (2005) Commercial Computer Games in the Australian Department of Defense Corti, K. (2006) Games-based Learning; a serious business application. Derryberry, A. (2007) Serious Games: online games for learning Diller, D., Roberts, B., Blankenship, S., Nielsen, D. (2004) DARWARS Ambush! Authoring Lessons Learned in a Training Game Hussain, T., etal (2010) Development of game-based training systems: Lessons learned in an inter-disciplinary field in the making Hussain T. & Ferguson, W. (2005) Efficient Development of Large-Scale Military Training Environments using a Multi-Player Game McGowan, C., Pecheux, B. (2007) Serious Games that Improve Performance Michael, D., & Chen, S. (2006) Serious games: Games that educate, train and inform Sims E., Salas E., Burke S. (2004) Is There a ‘Big Five’ in Teamwork Snider, M., Carley K., Moon, I. (2005) Detailed Comparison of America’s Army and Unit of Action Experiments Squire, K. & Jenkins, H. (2003) Harnessing the power of games in education • • • • • • • • • • 44

45. Thank You! Ross Kukulinski ross.kukulinski@asti-usa.com 45

46. Useful Links: www.seriousplayconference.com www.seriousgamesdirectory.com www.seriousgamesassociation.com Contact: sbohle@seriousgamesassociation.com