Sterling Somers : sterling@sterlingsomers Dr. Robert West : robert_west@carleton Both @

1. Macro Cognition : Using SGOMS to Pilot a Flight Simulator Sterling Somers : sterling@sterlingsomers.com Dr. Robert West : robert_west@carleton.ca Both @ Institute of Cognitive Science Carleton University Ottawa, ON, Canada

2. SGOMS • SGOMS is an extension of GOMS to deal with interruptions, teamwork, and re-planning • Adds a control structure above unit tasks – planning unit • SGOMS in ACT-R • A particular way to program ACT-R to do complex tasks • Adds more buffers to ACT-R • Planning unit buffer • Unit task buffer • Method buffer • These can be thought of collectively as the goal module • To test SGOMS we need to show that it works across tasks

3. Different SGOMS models in the lab Sandwiches

4. Why Aviation? • Real World • Dynamic • Both Planned and Reactive Behaviour • Replanning • Expertise • Continuous / Feedback

5. Interface • Simulator • FlightGear • Free • Open Source (hack simulation loop?) • Custom Python API • Property tree via UDP/HTTP/Telnet • http/telnet simultaneously (read/write) • State / Control of aircraft • Problem • Real time VS ACT-R time (for now…) • Simple Vision • Python bindings to OpenCV • Very efficient • Redundancy check possible

6. Vision • Top Down: • SOS Vision • Vision Module: • When not busy • Request from environment • Motion Detection • Python bindings OpenCV • Screen shot VS centre of screen comparison / frame Motion Detection Screen Shot at frame 1 Screen Shot at frame 2 (arrow indicates direction of movement)

7. Vision : Dynamic Salience / Context Dependence • Visionuses salience factor (0-1) to determine probability of seeing an environment item • Salience of Horizontal and Vertical movement calculated according to size of movement (in pixels ) • is pixel movement threshold • Possible different for different contexts?

8. Planning Unit: Prepare for take off Planning Unit: To take off speed Cue: Motor module: not busy Auditory Module: no engine noise + DM DM Unit Task: Get to speed Unit Task: Get Started Press Both Brakes Set Trim Start Engine To Full Throttle Read Air Speed Release Brakes + + + + + + Unit Task: Watch Speed Unit Task: Do settings Operator Pool To Full Throttle: Continue increasing throttle until lever stops moving Motor Methods

9. Planning Unit: To take off speed Maintain Heading DM Rudder, etc. + Unit Task: Get to speed Clear Buffer + (resulting in resume) To Full Throttle Release Brakes Read Air Speed + + + Interruption Unit Task: Watch Speed Vision Module Look at [target] Top-down request Look out window Opportunistic, when no top-down request See movement + Reasonable sized move

10. Interruption • Interruptions and re-planning take place on different levels • Case 1 • Bottom up scanning • Small • Expected • Context remains the same (still taking off) • Handle it within the unit task • Case 2 • Bottom up scanning • Large • Unexpected • Context is changed (abort take off) • New unit task needed • SGOMS Hypothesis – there is a set processing structure used across tasks

11. Thank you. Sterling Somers : sterling@sterlingsomers.com Dr. Robert West : robert_west@carleton.ca Both @ Institute of Cognitive Science Carleton University Ottawa, ON, Canada