Interactive Vehicle Level Human Perfomance Modeling

1. Interactive Vehicle Level Human Perfomance Modeling Presented by Mr. Tim Lee DCS Corporation 1330 Braddock Place Alexandria, VA 22302 Phone: 703-683-8430 X 203 Fax: 703-684-7229 tlee@dcscorp.com

2. Introduction • TARDEC Embedded Simulation Team and the Joint Virtual Battlespace (JVB) Program conducted a collaborative effort to develop a technique to implement interactive human performance models for the crew of simulated vehicles on a virtual battlefield, aka Vehicle Level Human Performance Model (VLHPM) • Initial development and integration based on TARDEC Vetronics Technology Testbed (VTT) Vehicle model Joint Virtual Battlespace Battlefield Friendlies Enemies Vehicle Model Sensors Actuators Automation System Architecture Human Performance Model Operators MM Interfaces Decision Aides HPM ICD JVB FOM Vehicle Level Human Performance Model

3. Introduction (continued) • VLHPM utilized ARL Human Research & Engineering Directorate’s Improved Performance Research Integration Tool (IMPRINT) by Micro Analysis & Design • Design event driven models (not scripted static models) • Utilize prioritized goals concept • TARDEC expanded the effort to model operators of Command Vehicle (CV) and Armed Robotic Vehicle (ARV) in FCS LSI’s Unmanned Combat Demo • Unmanned Combat Demo • Utilizing Crew integration & Automation Testbed (CAT) vehicle assets • Evaluate maturity of key robotics technologies, CV / ARV concept • Two CV operators; Two (or more) ARV operators • Experiment with different operator to robotic asset ratios to determine optimum

4. Introduction (cont.) • CAT vehicle limitations for UCD • Two man CAT crew • Experiments limited to two ARV operators; no simultaneous CV operations • Augmentation with constructive operator model • Four constructive operators, Two CV operators, two ARV operators, feasible • Low cost additions and modifications of crew, crewstation model • Low cost experimentation (vs with real HW, operator) • Mutual refinement and validation with real system

5. CV/ARV VLHPM Overview CV/ARV Vehicle Model CV/ARV Operator HPM Video (for monitoring) Embedded Simulation System IMPRINT 6.0 Runtime JVB C3 Grid Surrogate CV Dpty CV Cdr Simulation Control ESS state/mode control, CGF control AAR, data log control Visualizer Operational Commands ARV Operator 1 ARV Operator 2 CAT Vehicle Sensor simulation Sensor motion, visualization, automation (ATR simulation, auto tracking) Control Status Crewstation Interface ARV Operator 3 ARV Operator n CAT vehicle Weapon simulation Weapon motion, control, round Flyout COM Execution Time Control CAT Vehicle Mobility Simulation Automotive simulation, auto-drive Navigation sim DIS / HLA Interface Vehicle I/F Route/ Tactics Planner Communications Simulation JVMF message generation, reception WSTAWG OE UGV Simulation UGV/ARV mobility, tele-op, RSTA, OCSW, Javelin simulations Internal OTB Modified from VTT Model New Stuff External SAF Environment DIS / JVB / RPR Disabled when interfaced to external SAF Optional Stuff

6. CV/ARV Vehicle Model • Extensive modifications and additions to baseline model of VTT Embedded Simulation System (ESS) • Automation intensive CAT architecture • Multiple UGV simulation • Stryker variant for CV • Wheeled vehicle mobility • Concept 5.5 ton wheeled ARVs with configurable payload • Weapon simulation based on Cougar turret (OCSW, Javeline) • Simulation of RSTA sensor, Target tracking sensor • Semi-autonomous mobility Example screen shot of vehicles. Not from actual experiments

7. CV/ARVHuman Performance Model • IMPRINT Goal oriented task network paradigm • Goals can be prioritized • Execution of tasks (task networks) to achieve a goal can be interrupted by execution of tasks of a higher priority goal • Behavior models designed for real-time interaction, not static analysis • Operators are never loaded more than 100% • Reactions to external stimuli represented as mutually exclusive prioritized goals • Continuous main mission tasks that monitor for external stimuli • Awareness and Interaction via MMI • Assume plausible level of automation • Processing of audio visual input aggregated as a task load factor for each task • Operations that require sight are approximated with, assisted by, or replaced by automation that provide digital data (route / tactics planner, Automated Target Recognition, Automated Target Tracking).

8. CV/ARVHuman Performance Model: ARV Operator 7 Report In Defilade Goal -Tell CDR “In Defilade” 6 View RSTASCAN Goal -View Image - Send Resume Msg ARV In Defilade Report ARV RSTA SCAN • 2 • View RSTA Target Goal • View Target • Add Target to Queue • Tell Cdr “Targets” Monitor ARV 1 Teleop ARV Goal - Teleop ARV back 20m -Update ARV Route -Send Resume Msg ARV ATR Alarm ARV Stuck Alarm • 4 • Engage Threat Goal • -Confirm Targets • -Engage Targets • Clear Targets • -Tell CDR “Engagement • Complete” 5 Plan to Objective Goal - Develop ARV Plan -Send ARV Plan “Engage” & ARV Targets>0 “Plan to Objective” Monitor Internal Communications 3 Route to Defilade Goal -Enter ARV plan -Send ARV plan “Defilade” ARV Operator Mission and Goal Breakdown

9. CV/ARVHuman Performance Model: CV Commander 1 Process C2 Message Goal -Enter CV Route -Tell ARV “Plan to Obj” -Set Fire Perm.= Free -Tell DPTY “Drive Route” C2 Alarm (Unit March) Monitor External Communications 5 View Threat Goal -Confirm Targets -Report Type=SPOT CV ATR Alarm & !FREE Monitor Surveillance Systems • 2 • Seek Defilade • Goal • -Update CV Route • Say “Defilade” “Targets & ARV Targets>1 CV ATR Alarm & FREE 4 Engage Threat Goal -Set Fire Perm.= Free -Confirm Targets -Engage Targets -Report Type=SIT ARV Targets>1&Free &”In Defilade1” &”In Defilade2” & “At Destination” OR ARV Targets=1 & FREE 3 Unit Engage Goal Say “Engage” Monitor Internal Communications “At Destination” 7 Send C2 Report Goal -Enter C2 Report -Send C2 Report Detonation & CV Targets>1 6 Resume Mission Goal -Update CV Route -Tell ARV “Plan to Objective” -Report Type=SIT “Engagement Complete” Monitor Self Protection Systems CV Commander Mission and Goal Breakdown

10. CV/ARVHuman Performance Model: CV Deputy/Driver 1 Drive Route Goal - Steer to Waypoint -Accelerate/Brake to Waypoint -Say “At Destination” “Drive Route” Monitor Internal Communications CV Driver Mission and Goal Breakdown

11. CV/ARVHuman Performance Model Example Task Network: Driving Task network

12. CV/ARVHuman Performance Model: Automation • Both “manual” and autonomous control of mobility designed to follow a route generated by a route / tactics planner • Considers terrain elevations • Considers terrain type • Manual control model adjusts actuator values in real time to keep vehicle on route • Autonomous control model issues a “plan” that contains the route and RSTA scan commands to the vehicle model • Target detection by RSTA Automatic Target Recognition • Target engagement with Automatic Target Tracking

13. CV/ARVHuman Performance Model: Automation Open Terrain Non-traversable area Roads Routes for two ARVs And CV Non-traversable area Pictorial representation of example terrain type data and generated routes Elevation data is not shown

14. CV/ARVHuman Performance Model: User Interface • Real time display of Visual Auditory, Cognitive, Psychomotor (VACP) loading on each operator • Real time indication of vehicle position navigation data • All interfaces and data logging of IMPRINT 6.0

15. Experiments • Unmanned Combat Demo • One CV, two ARVs • Initial integration demo of maneuvers • Capstone Demo • One CV, one ARV • Limited participation due to risk mitigation • Virtual Distributed Lab for Modeling & Simulation (VDLMS) First Application • One CV, one ARV • Modified vehicle model, and HPM for participation as a Forward Observer / Laser Designator of “Netfires” concept evaluation • Maneuver in recon mission; • Spot report or call for fire (for Precision Attack Missile) • Perform laser designation • Participated in majority of recorded runs simultaneously with CAT crewstations

16. Lessons Learned • Real-time interactive simulation of complex behaviors of combat vehicle operator(s) is feasible • With goal oriented task model of IMPRINT • With a high fidelity vehicle model that has a rich, portable data interface • With sensor automation such as ATR, ATT that minimizes the need to convert and process visual information • Enhancements to IMPRINT can simplify dynamic HPM development, such as: • Nested IF or While constructs • Temporary variables in macros • Improved External Model Call facility • Inter-operability with cognitive models (SOAR, ACT-R) • Improvement of command data protocol / interface is needed • JVB C3 grid interactions afforded limited flexibility • Direct manual command injection via a “C3 Grid surrogate” was implemented for non-JVB experiments • Additional manual input of decision thresholds for target engagement were added for First App

17. Future Plans • Enhancement of C3 interactions, processing, decision making • Higher echelon platform(s) or organization • Improvement of C3 interaction protocols, data sets • Run-time manual command input • Simulation C2 interface: JVB, Netfires, MATREX • Modeling of or integration with other automation or simulation tools • On board decision aids • Higher fidelity component or functionality models of VDLMS (other HLA federates) • Integrated Unit Simulation System (IUSS) of SBCCOM • Cognitive simulation tools (ACT-R, SOAR)

18. Future Plans • Higher fidelity route / tactics planner • Higher fidelity environment data • More rigorous algorithms • Controlled data collection and analysis • Manned crewstations & HPM executing same operation (achieved at First App, but not enough correlated data) • Compare HPM “predicted” performance with actual • Adjust HPM • Continued support of simulation analysis objectives of FCS SDD • Enhancements based on all the above • CAT HPM developments in parallel with crewstation enhancements / modifications • Unmanned Combat Demo II