ADVANCED DISPLAYS FOR DISMOUNTED WARFIGHTERS 21-22 SEPTEMBER 2010

1. UNCLASSIFIED ADVANCED DISPLAYS FOR DISMOUNTED WARFIGHTERS 21-22 SEPTEMBER 2010 UNCLASSIFIED

2. Outline • Introduction • Types of Advanced Displays • Current Displays and Lessons Learned • Occluded Helmet Mounted Displays • Head Mounted Displays • Fused Night Vision Goggles • Conformational Displays • Tactile Displays • Flex Displays • See-through Helmet Mounted Displays • Augmented reality displays and Enhanced Cognition • Conclusions

3. Introduction The purposes for using displays are: • to present or hold up to view • to provide information or graphics on a screen • to provide a representation of information Now just a minute, earlier you said x equals 4! x+3=5 X=2

4. Introduction The purpose of advanced displays is to immerse warfighters in the operation so they can experience and convey critical information from real-time data feeds in an intuitive, recognition-based manner.

5. Multifunction Displays (MFDs) Introduction Definition- a display surface which, through hardware or software, is capable of displaying information from multiple sources, in several different reference frames. It may display different groups of data one at a time or in a combined fashion. Multifunction, not multiple displays

6. Types of Advanced Displays Headmounted Body-worn or Carried Formed Configurable Occluded See-through HMDs GMDs Augmented Reality Displays Tactile Displays PDAs Fused NVGs Flexible Displays Notebooks

7. Current Displays and Lessons Learned Current Systems Color Helmet Mounted Display FBCB2 AN/PVS-14 NVG • The Soldier is the most difficult “combat platform” to interface with! • One Size never fits all • Soldiers have different opinions • Soldier Acceptability is critical

8. Introduction Current Displays and Lessons Learned Land Warrior 2005 Land Warrior “Manchu” 2007 Land Warrior “Strike” 2008/2009 Nett Warrior 2012/2017 The Future – Nett Warrior All BCTs • Command & Control • - Soldier: No capability • - Leader: Voice & Data • Key Attributes • - Weapon: Dropped • - Radio: EPLRS • - Interoperability: - UGV interoperability for • IED Defeat • - Air/Ground Integration thru SADL • Size & Weight • - 7 components • - 9 pounds • Command & Control • - Soldier: Voice & Data • - Leader: Voice & Data • Key Attributes • - Weapon: Subsystem • - Radio: EPLRS • - Interoperability: w/ FBCB2 • Size & Weight • - 11 components • - 16 pounds • Command & Control • - Soldier: No capability • - Leader: Voice & Data • Key Attributes • - Weapon: Dropped • - Radio: EPLRS • - Interoperability: w/ limited external • assets • Size & Weight • - 9 components • - 11 pounds • Command & Control • - Soldier: Interoperable with Soldier Voice & PLI • - Leader: Digital Voice & Data • Key Attributes • - Weapon: Dropped • - Radio: EPLRS/JTRS (P3I) and Open • Architecture • - Interoperability: increased w/ external sensors • (JBC-P and Fire Control Systems) • Size & Weight • - Minimum essential • - 10 pounds (Objective) Evolution From Land Warrior To Ground Soldier System (Nett Warrior): Current Path to Modernization

9. Introduction Current Displays and Lessons Learned Animation Go to Undo Icons • Cognitive Walkthroughs with Target Audience Soldiers • Several Iterations of User Juries for the Ground Soldier System • Ease of training • Intuitiveness • Speed • Errors • Limited Hardware Evaluations

10. Occluded Head Mounted Displays • Currently planned for many systems (NETT Warrior, Common Controller, etc.) • Lightweight • Relatively high resolution and easier to read • Require less power than many portable devices • Less glare than many portable devices • Hands free

11. Occluded Head Mounted Displays POTENTIAL PROBLEMS • Both real world and HMD imagery must be within the user’s depth of field, dark focus and dark vergence • Restricted field of view can impair performance and adversely affect the ability to see the “whole picture” and reduce SA • Binocular rivalry • Competition for the attention of the wearer/attentional tunneling which adversely affects dual task performance • Eyestrain and blurry vision • As HMD wearers move their heads, displayed objects in front continue to be in front and this is unnatural • Motion sickness, often explained as being caused by sensory conflict. • Interference with night vision devices and weapon sights • Fogging due to weather • Potential for placing the musculoskeletal system of the head and neck under increased levels of stress

12. Occluded Head Mounted Displays Helmet Mounted Displays • Scaling Robotic Displays: Displays and Techniques for Dismounted Movement with Robots • Soldiers performed significantly worse with the GMD than they did with the HHD on course completion times, driving errors, and the number of times they drove off course. • Soldiers also preferred the HHD to the GMD and rated the workload with the HHD lower.

13. Occluded Head Mounted Displays Dismounted Land Navigation Study Helmet Mounted Displays PTN Tactile System: GPS sensor, electronic compass, processing unit,battery pack,eight tactors placed equidistant,on a belt around the torso,worn over T-shirt LAND WARRIOR HMD GPS SYSTEM HANDHELD GPS PLGR • 1. 24 Soldiers (N = 21) • 2. 3 routes, each with 2 waypoints & endpoint, each approx 1800m • 3. 3 systems: Tactile, PLGR, LW HMD • 4. Order of routes and systems counterbalanced

14. Occluded Head Mounted Displays Helmet Mounted Displays Dismounted Land Navigation Outcomes Target Detection

15. Occluded Head Mounted Displays Very good Good Somewhat good Neutral Somewhat bad Bad Very bad Helmet Mounted Displays Dismounted Land Navigation Soldier Feedback

16. Occluded Head Mounted Displays Fused Night Vision Goggles Urban Enhanced Night Vision Goggle Enhanced Night Vision Goggle

17. Occluded Head Mounted Displays Scene Interpretation Fused Night Vision Goggles OVERLAY COLOR • Contrasting color for FLIR • image vs. green I2 image • improves target detection • speed and range. • Bonnett, Redden, & Carstens, 2003 • Contrasting color also • assists in differentiating • terrain characteristics.

18. Occluded Head Mounted Displays Fused Night Vision Goggles High Thermal Target Detection Optimal Mix of I2 and Thermal Depends on Purpose Patrolling Navigation High I2

19. Urban Enhanced Night Vision Goggles (UENVG) Urban Enhanced Night Vision Goggles (UENVG) • UENVG is a prototype system used to determine the impact of adding a Short Wave InfraRed (SWIR) capability into the existing ENVG system. • SWIR • works in darker conditions than I2 • can see objects with great clarity on moonless nights because night sky radiance (nearly all in SWIR wavelengths) emits 5 to 7 times more illumination than starlight ) • is not visible to the human eye but interacts in a similar manner as visible wavelengths • has shadows and contrast and can see through glass

20. Occluded Head Mounted Displays Flexible Displays • Near term Objectives • Compare indoor and outdoor sunshine • readability • Collect Soldier opinions on the technology • Recent Efforts • Evaluations at Fort Benning (2009, 2010) • Handheld and Forearm • Instructors from Warrior Training Center • Evaluation at Fort Bliss (2009) • Survey of9 prototypes • Benefits • Much less glare, lighter, rugged • Much less power consumption • Longer battery life

21. Conformational Displays Flexible Displays Far-term vision: Novel form-factors Near-term: Rugged, Low Power, Compact, Lightweight Enabled by FDC’s Unique GEN II (37x47cm) Pilot Line toolset and People 21 Partners Representing: Display Technology Manufacturing Tool Suppliers Materials Developers Defense Contractors

22. Conformational Displays Tactile Displays Scalability of Robotic Displays: An Evaluation of Controller Display Options • Evaluated 3 options for TALON controller Display: • -6.5 inch Split Screen • -3.5 inch Toggle Screen • -3.5 inch Toggle Screen plus Tactile belt • Results • 3.5 Toggle Screen associated with slower • performance and higher workload • than either (A) or (C). No difference between • (A) and (C) • Tactile belt enabled a smaller screen while • providing cues that supported performance.

23. Conformational Displays Tactile Displays 3.5”Toggle Display • 6.5” Split Screen 3.5” Toggle Display with Tactile Belt

24. Conformational Displays Tactile Displays • Subjective Measures • Task difficulty and SA ratings were worse for the toggle display. • Some Soldiers preferred the toggle/tactile display because they felt they were able to pay more attention to the driving display while wearing it. • Others preferred the split screen display because of the wealth of information provided. • A fixed position camera and latency cause some driving difficulties. • The egocentric GPS caused problems when Soldiers glanced away from the screen.

25. See-through Displays Augmented Reality Display • The Last 18 Inches: How can you improve perception, squad coordination, and decision making in tactical, high stress urban operations? • Squads and platoons are not well served by current information systems • Overloaded: too much 2D, 3D, video data • Confusing: irrelevant data, old, out-of-scale, inaccurate • Optimized for strategic levels, not tactical block-to-block, not flowing from bottom-up • Its getting worse: soldiers/vehicles as sensors, geospatial & human network models • Most equipment today distracts during operations • Pain in the neck: heavy, cumbersome, ill-suited for on the move • Can’t make use when and where needed, intended for desktop • Need for “on the go” interfaces, computing, sensing • Independent module development, proliferation of multiple methods (the TV remote problem) • High stress tactical decision making • Hard to remember important stuff • Too much to pay attention to • Need tools to aid attention and memory

26. See-through Displays Augmented Reality Display “I’ve got your back!” Squads Need Omniscience & Telepathy within the City Block • Extend superiority into short-range combat inside urban and jungle environments. • Extend collaborative planning, rehearsal, and execution capabilities from company to squad level. • Enable quieter, non-linear, distributed, increased op tempo, 3D operations (e.g. take down building from 3 directions) • Enable effective, quick dynamic replanning • Reduce chaos, fratricide, avoid surprises

27. See-through Displays Augmented Reality Display • Make data useful to platoon/squad/soldier in context • Geospatial registration of annotations • Agent-based systems to recognize intent, state • Tools to leverage models for preview, execution, debrief • Relevance at the level of seconds and meters • Make communication for squads and platoons more effective • Integrate planning, execution, and debrief in one system • Improve remote (higher command) understanding of local ops/data/history by virtual experience • Make interaction with information “on the go” • Overwatch feeding/filtering information selectively • Interface sensors respond to natural body actions, non-screen interfaces • Advances in smaller, lighter, lower power, higher resolution displays, computing, sensing enable head/body worn approaches • Make overwatch automatic • Plan via analogy • Fuse, filter, prioritize information automatically (learn)

28. See-through Displays Augmented Reality Display • Civilian technology offers examples to learn from and leverage… • Madden Board and first down line – graphics registered in scene images for enhanced communication • Nintendo Wii as “natural interaction” • NFL offensive play caller – in booth above field – overwatch feeds info to coach and QB • Nascar crew chief communication to driver • “Mission Impossible” controller feeding data and info to agents in mission/field • Blackberry, iPhone, iPod, HUD (Private Eye) displays and interfaces • Social information systems like GoogleEarth, Facebook, Wikipedia, wwmx.org, rich info sources with engaging interfaces • GoogleEarth is an example of local users creating overlays on global 3D base model maintained/updated centrally

29. Conclusions • The Dismounted Warfighter is the most difficult customer for displays. • Display technology continues to advance and today’s failures may be tomorrows successes. • Displays should be chosen based on mission requirements, echelon level and on environmental considerations. • Human factors considerations and experimentation are critical for effective display design.