Simulator Adaptation Syndrome Discussion

1. Simulator Adaptation Syndrome Discussion Dr. Michael A. Mollenhauer Realtime Technologies 6/12/2003

2. Agenda • What is Simulator Adaptation Syndrome? • Measurement of Symptoms • Visual and Vestibular Systems • Simulator Design Issues • Simulator Application Issues • Potential Countermeasures

3. What’s the Problem? • Rates of Occurrence • 20 – 40% Fighter Pilots Drop Out of First Training Simulator Flights • 30 – 50% Moderate Symptoms in Ground Vehicle Simulators • 40 – 60% Older/Experienced Drivers in Ground Vehicle Sims • Implications for Validity of the Results, Participant Motivation, and Product Appeal • Do No Harm!

4. What is Simulator Adaptation Syndrome (SAS)? • Ill Feelings Associated With The Use Of Simulation Devices • “Syndrome” Because Both Polygenic And Polysymptomatic • Causes: Discussion To Follow • Symptoms: Eye Strain, Headache, Postural Instability, Sweating, Disorientation, Vertigo, Pallor, Nausea, And Vomiting

5. Motion Sickness vs. SAS • Different Causation • Motion Sickness – 0.2 – 1 Hz Whole Body Oscillation, Particularly Up and Down • Can Get Motion Sick Without Seeing Anything • Can Get SAS Without Any Movement –> Fixed Base Simulators • SAS Requires Some Feeling of Vection • Wired to Same Response Processes In Sympathetic Nervous System • Must Have Working Vestibular System To Get SAS or Motion Sickness

6. Who Is Affected? • Older More Than Younger • Female More Than Male • Asian More Than Caucasian • Experienced More Than Novice

7. Theories of Simulator Sickness • Cue Conflict Theory • Poison Theory • Theory of Postural Instability

8. Cue Conflict Theory • Symptoms Caused by Mismatches Between Sensory Expectation and What Actually Occurs • Eyes Indicate Movement, Vestibular System Does Not • Problems • Doesn’t Identify Why Mismatch is Bad • Cannot Explain Adaptation • Why Doesn’t It Occur With Every New Activity/Movement

9. Poison Theory • Evolutionary Point of View • Blurred Vision, Sensory Conflict, Uncoordinated Movement Indicate Recent Poisoning or Intoxication • Natural Response is to Remove Poison Through Emesis • Problems • No Predictive Capability • Does Not Explain Many Aspects of SAS Including Adaptation, Age, and Experience Effects • No Accounting for Individual Differences

10. Theory of Postural Instability • Sensory Systems Are Constantly Trying To Maintain Stability • Sickness Comes Attempting To Stabilize Under Conditions Where Strategies Have Not Yet Been Developed • Postural Instability Is Requirement Preceding SAS • Accounts For Adaptation, Experience, Etc. • Problems • Why Does Emesis Occur • Why Is A Lack Of Strategy A Bad Thing

11. Measurement of Symptoms • Simulator Sickness Questionnaire (SSQ) • Kennedy, R., Lane, N., Berbaum, K., and Lilienthal, M. (1993). • 3 Subscales and Composite • Nausea Subscale • Oculomotor Discomfort Subscale • Disorientation Subscale • Total Severity • Scoring Formulas • Postural Instability • Measured With Force Plate • Individual Differences Pre-Exposure

12. Questionnaire Format Example

13. Pertinent Human Sensory Systems • Visual System • Vestibular System

14. Important Aspects of Visual System • Central vs. Peripheral Vision • Optic Flow • Perception of Depth • Movements of the Eye

15. Central Vision Answers “What Is It” Small Stimulus Patterns, Fine Detail Central Retina Only Well Represented In Consciousness Object Recognition And Identification Peripheral Vision Answers “Where” Large Stimulus Patterns Image Quality And Intensity Not Important Peripheral And Central Retina Not Well Represented In Consciousness Spatial Localization, Orientation, And Motion Central vs. Peripheral Vision

16. Optic Flow • Perceived From Movement Of Objects In Optic Array • Locomotor Flow Line Derived From Flow Of Objects Beneath Observer • Used To Determine Current Path Of Travel • Implications For Geometric Accuracy Of Displayed Images

17. Locomotor Path Determination

18. Perception of Depth • Oculomotor Cues • Given By The Position Of Our Eye And Tension On The Muscles Within The Eye • Binocular Cues • Slightly Different Scenes Are Formed On The Retina Of Our Eyes • Pictoral Cues • Can Be Deduced From Looking At Still Picture • Motion Cues • Deduced By How Objects Appear To Move When Observer Moves

19. Oculomotor Depth Cues • Proprioceptive Cues Interpreted By The Brain • Convergence • Inward Angular Positioning Of The Eyes To Keep An Object Focused On The Fovea As The Object Is Moved Closer To The Observer • Range: 0 – 14 Ft • Accommodation • Flexing Muscles In The Eye To Change The Shape Of The Lens As An Image Is Brought Into Clear Focus • Range: 0 – 5 Ft

20. Binocular Depth Cues • The Eyes See The World As Two Slightly Different Pictures Due To The Slightly Different Vantage Points Created By The Distance Between Them • The Brain’s Ability To Fuse These Disparate Images Into A Single Visual Image Produces Strong Perception Of Depth

21. Pictoral Depth Cues • Pictorial Cues Give The Illusion Of Depth To Two-dimensional Art • Brain Uses Pictorial Cues To Turn Two-dimensional Image On Retina Into A 3D Image • Absence Of Cues Can Cause Uncertainty

22. Pictoral Depth Cues • Relative Size • Smaller Objects of Same Shape Appear Further Away • Interposition • Obscured Objects Seem Further Away

23. Pictoral Depth Cues • Texture Gradient • Detailed Texture Appears Closer to Observer

24. Pictoral Depth Cues • Linear Perspective • Parallel Lines Converge at Greater Distances from Observer

25. Pictoral Depth Cues • Shades and Shadows • Orientation and Intensity of Shadows Convey Depth Information • Relative Height • The Higher the Object in a Viewing Plane, The Further Away it Appears

26. Pictoral Depth Cues • Atmospheric • The “Washing Out” of Objects and Object Detail Due to Haze Makes them Appear Further Away

27. Motion Depth Cues • Objects That Are Further Away Appear To Move Slowly In The Direction Of The Observer’s Movement • Closer Objects Appear To Move More Rapidly In The Direction Opposite The Observer’s Movement • Accretion And Deletion Are Related To Motion Parallax And Interposition – Object That Moves To Cover Another Is Closer

28. Movements of the Eye • Consist of Saccade, Smooth Pursuit, Vergence, Opto-Kinetic Reflex (OKR), and Vestibulo-Ocular Reflex (VOR) • OKR and VOR Work to Stabilize Image on Retina • OKR Evaluates Image on Retina to Identify Slip, If Found it Triggers an Opposite Saccadic Eye Movement • VOR Has a Similar Goal But is Triggered By Information from the Vestibular System • More Discussion of These Provided Later

29. Important Aspects of Vestibular System • Responsible for Identifying Orientation and Acceleration of Head • Drives Balance, Motor Control, and Eye Movements • Semi-Circular Canals • Utricle and Saccule • Vestibulo-Ocular Reflex • Very Fast Operation (< 10 ms latency) • Body’s Sole Source of Immediate Acceleration Information

30. Semi-Circular Canals • 3 – Each Oriented To Detect Motion In Each Of The Three Planes In Which Motion Can Occur • Fluid In Canal Flows During Motion Which Stimulates Tiny Hairs On Internal Walls • Sensitive To 0.1 Deg/S2 • Good For Sensing Change Rather Than Sustained

31. Utricle and Saccule • Functions Similar to Semi-Circular Canals - Otolith • Utricle Oriented to Detect Acceleration in Lateral Direction • Saccule Oriented to Detect Acceleration in Longitudinal and Vertical Directions • Responsible for Orientation With Respect to Gravity

32. Otolith Illusions • Somatogravic Illusion • Lon Accel/Decel Causes Pitch Up/Down Sensation • Vertical Accel/Decel Causes Backward/Forward Tilt • Oculogravic Effects • Somatogravic Illusion Causes Altered Visual Perception – we actual see pitch down / up • Vestibular System is Trying to “Lead” Vision

33. Vestibulo-Ocular Reflex • Information About Head Movement Is Supplied To Visual System • Visual System Interprets And Makes Corresponding Eye Movement To Stabilize Image • Shaking Paper Vs. Shaking Head • Very Fast Acting • Adaptable – VOR Will Adjust It’s Gain To Support Different Sensory Arrangements

34. VOR and OKR • Work Together Synergistically To Stabilize Retinal Image • VOR Acts Fast And Corrects Well For 1-7 Hz Movement • OKR Responds Slower And Corrects Well For < 0.1 Hz Movements • Tight Feedback Loop Where OKR Helps VOR Adjust Gain And Therefore Adapt

35. VOR Adaptation • VOR Found To Adapt To Changing Visual Magnifications And Correlated To Sickness • Individuals Who Adapt Faster Are Less Likely To Experience Sickness Symptoms • The Greater The Change From “Normal”, The Longer the VOR Adaptation Process • Inconsistent Sensory Mapping Prolongs VOR Adaptation

36. VOR Adaptation • Visual Anomalies Can Alter OKR Feedback Which Also Hinders Adaptation • Large Individual Differences • Plasticity Of VOR or It’s Ability to Adapt Decreases With Age

37. Visual Perception of Self-Motion • Vection – Compelling Feeling Of Self-motion While Static With The Environment • Generated From Movement Detected In Optic Flow • Normally Accompanied By Some Vestibular Sensation – If Not Then Sensory Conflict • Strength Of Vection Increased By • Larger Movement Stimulus – Larger FOVs • Faster Optic Flow - Higher Relative Speed

38. Challenges of Driving Simulation • High Flow Rates – Relatively Low Altitude • More Flow Due To Proximity Of Buildings, Signs, Other Traffic, • Often Require More FOV • Higher Sensitivity To Environment/Vehicle Interactions • Application Is All About Vection, Vection Is A Precursor To Sickness

39. Simulator Design Issues • Display Flicker • Field of View • Geometric Display Design • Binocular / Monocular Viewing • Resolution / Aliasing • Display Refresh / Update Rate • Motion Cueing • Controls / Feedback • Transport Delay • Calibration • Head Mounted/Slaved Displays • Environmental Conditions

40. Display Flicker • Why 60 Hz or Better? • Perception of flicker interferers with saccadic eye movements, causes ocular muscle fatigue and eye strain • Linked to simulator sickness • Critical Fusion Frequency (CFF) is around 40 – 60 Hz in dark, foveal viewing conditions

41. CFF Depends On • Display Luminance • Brighter Displays Decrease CFF • Environment Illumination • Brighter Environment Decreases CFF, Interharmonics • Refresh Rate • Slower Refresh Decreases CFF • Eccentricity • Periphery More Sensitive, Larger Displays Decrease CFF • Age • Young People More Sensitive To Flicker • Gender • Women More Sensitive To Flicker

42. Field of View (FOV) • Long Implicated As An Exacerbating Factor • More Peripheral Stimulation Results In More Vection, Vection Linked To SAS • Performance Implications • Narrow FOV Results In Poorer Lane Keeping, Less Accurate Perception Of Speed, Less Eye/Head Movement, And Less Immersion • Effects Level Out At Around 140 – 160 Degrees • Implications for Validity • Depends On The Task

43. Geometric Display Parameters • Issue: How Well Display System Represents Real Viewing Conditions • Parameters: Eye Height, Angular Ratios, Viewer Orientation To Displays • Impacts Optic Flow Perception, VOR Adaptation, Task Performance • The Less Congruent With “Reality”, The Longer The Adaptation -> Potential SAS

44. Geometric Display Parameters • Inaccurate Cueing • Driver Not Oriented Appropriately May Feel “Skidding” • Eye Height Too High Results In Inaccurate Speed Perception • Display Demagnification Can Result In Inaccurate Steering Behavior • Flat Displays – Viewer Reports Initial Translational Movement During Turn Rather Than Yaw Rotation • Display Distance • Vergence And Accommodation • No Absolute Threshold But More Is Better Up To 15’ • Performance Trade-Offs Due to Distance • Suggested Minimum Is 40”

45. Binocular/Monocular Viewing • Stereo Displays Provide Depth Cues Through Binocular Disparity And Vergence • Active Stereo • Two Images Rendered In Each Frame Representing The Perspective View Of Each Eye • Shutter Glasses Are Used To Alternate Between The Corresponding Views • 96 Hz / 48 Hz Per Eye • Passive Stereo • Uses Polarizing Filters To Accomplish Same Task

46. Binocular/Monocular Viewing • Stereo Provides Positive Impact On Task Performance Provided Some Element Of Task Requires Viewing Within 15 Feet Or So • Appears To Elevate Risk Of SAS By Increasing Vection And Reducing Vection Onset Times • May Potentially Cause Eye Strain If Not Adjusted Appropriately – Interpupillary Distance, Focal Distance, Etc. • Costs – Equipment, Performance, Brightness

47. Resolution / Aliasing • Resolution • As Resolution Degrades The Eye Attempts To Accommodate The Image Causing Eye Strain • FAA Requires 3 Arcmin/Pixel Or Better For Flight Training Simulators • PD5000(?) - 2.3 Arcmin/Pixel, Around 20/40 – 20/50 Vision • Likely Impact On Task Performance (Sign Legibility, Far Target Detection, Etc.) • Likely Impact In Oculomotor Discomfort Portion Of SAS If Resolution Too Low, May Slow VOR Adaptation, May False-Trigger OKR • Aliasing • Causes Apparent Motion In Scene, Can Be Interpreted As Flicker • Changes Eye Scanning Strategy

48. Display Refresh / Update • Display Refresh • Potential For Perception Of Flicker Or Detection Of “Smearing” Or “Ghosting” • Most CRTs Fast Enough • LCDs – Phosphor Decay Issues, Smearing, Ticking – Can Trigger OKR • Graphics Update • Function Of How Fast Simulator Subsystems Process • Variable Rates May Slow VOR Adaptation Processes • VOR and visual processing approx 10 – 20 ms. • Slow Rates Appear “Jerky” And Could Be Perceived As Flickering

49. Motion Theory • Scaled Representation Of Dynamic Forces –Normally 10-50% Of Normal • In 6 DOF Systems, Onset Provided By Translation, Sustained Provided By Tilt • Washout Algorithm – Always Returning Motion Base To Center Below Levels Of Perception • Always Some Amount Of Error And Some Amount Of Transport Delay • Affects Skills Based Driving Behavior More Than Knowledge Based Behavior

50. Motion Configurations • Electric vs. Hydraulic • Degrees Of Freedom • Operating Envelope • Frequency Response • Visuals On And Off Motion