Intelligent Integration The Next Generation of Intelligent, Automated Systems

1. Intelligent Integration The Next Generation of Intelligent, Automated Systems Jeffrey Wallace and Barbara Hannibal 3rd Annual San Diego Regional Security Conference September 14-15, 2010

2. About Us • Unique Systems Engineering and Development • A distillation of over $2B in government R&D • Partnered with OSD, JFCOM, “Team Orlando”, the 4 services, SRI, Academia, and others • Solved world-class/grand challenge problems on F-35, CVN-21, healthcare IT, etc.

3. Underlying Technology Thrusts • “Nervous System” – component activation, integration, and interoperability • Knowledge representation and reasoning • Conceptual graphs • Problem solving module integration • Complex context representation along with reasoning integration and problem solving • Ontologies and other Semantic Web technologies

4. Problems • Little to no time for critical decisions • Imperfect information, must adapt to “dirty” environments • Interactions happening faster than humans can react • Volume of information, battlespace clutter, and increasing speed: DE, hypersonics, UxVs • Current Command and Control (C2) is not real-time and the only source of information

5. Example: Anti-Ship Missile Defense UAS CIWS Chinese Anti-Ship Missile DF-21 SBX M-Ship • Increase Overall System Speed and Effectiveness • Simplify Installation and Maintenance DDG

6. Summary Capture needs through application of novel techniques Creation of complex, realistic, and scalable networks of component inter-relationships Distribution of autonomous controls and monitors Implementation of complex webs of cause and effect Dynamic alteration of the component execution structure Adaptation and evolution of the system 8

7. MIW ASW IBGWN SUW Fast and Flexible 100011100110011010101010101010011110010010101001010 1010101010101010011110010010111001010001010 001101011101010100101010111101000011110001111001011101011100 010101000110101110101010010101011110100001111000111100101110101110001010 011001111000010101000110101110101010010101011110100001111000111100101110101110 0111100101110101110001010 101000011110001111001011101011100010100011 10100001111000111100101110101110001010 10100001111000111100101110101110001010001111100000111110101 Adaptive Mesh 10100001111000111100101110101110001010001111100000111110jfdl;alskedj101

8. Code Generation ExampleExample: Turret/Fire Control Process Firing Commands (and Queuing Them) Slew Elevate Fire When Slew and Elevate are Complete

9. Code Generation ExampleTurret: Fire Command void Turret::fire() { P_VAR P_BEGIN(2) // Wait until the turret movement is completed WAIT_FOR(1, slewComplete, -1); WAIT_FOR(2, elevateComplete, -1); // Fire the weapon, this would activate the real gun Fire_M256(); RB_cout << "Flash, Boom, Bang, Echo" << endl; fireComplete = 1; P_END }

10. Demonstration Main Points • Web and Real-time ESB Cooperation • RT ESB handles sustained ~ 1000 messages/sec with modest hardware and complex systems • Multiple, simultaneous inter-processor comms employed (shared mem, TCP/IP, RS-232 at 19,200 bps, UDP Broadcast) • Great performance of complex applications (physics-based 3D visualization and motion prediction) • Large number of diverse databases and data types employed

11. Demo Information Flow Battlespace Network Predominantly real-time C4ISR Network Web-based/Non-real-time Shadow UAV System Components Sensor View 1 C2PC C2PC Gateway Sensor View 2 Battlespace Entity Generator Web Message Service C4ISR Network Interface (RS-232) RS-232 Interface

12. Airframe Stability Augmentation System Pilot Controller Electro- Mechanical Systems Comm Uplink and Downlink Control Station Interface Suspension Wheels/Tires Powerplant BIAS Application Interface UAV System

13. Physics-based Imager Model Open Scene Graph (OSG) *3D Modeling* OSG nVidia Plugin Electro- Mechanical Systems Modeling Open Producer *3D Rendering* CONDOR Processing Engine Physics-based Atmospheric Modeling Open Threads CONDOR External Application Interface Composability Example: EO/IR Camera IR Camera Simulation

14. The Problem Most “intelligent” computer applications use rule-based expert systems (or some variant) as a means of storing expert knowledge, however these systems always have one or more severe limitations that make them unsuitable for future intelligent software development…….. BRITTLE……….ONCE PROGRAMMEDTHEY ARE EXTREMELY HARD TO MODIFY COMPLEX……….HARD (Expensive) TO MAINTAIN SLOW……….CAN’T APPLY LARGE AMOUNTS OF KNOWLEDGE FOR REAL TIME APPLICATIONS RIGID……….CAN’T ADAPT OR SCALE WITH CHANGES IN SITUATION The Conceptual Graph approach was designed to overcome all of these common problems

15. KNOWLEDGE STORAGE USING CONCEPTUAL GRAPHS New conceptual modeling approaches allow knowledge to be represented in an “Object-Relationship” graphical format - A surprisingly high level of knowledge complexity can be represented using only three components: Actors Concepts Relationships Une Chat assis sur une matte A Cat sits on a mat STAT LOC CAT SIT MAT

16. CONCEPTUAL REASONING ENGINE (CORE) Stored Knowledge (Reusable) Observer Actor INPUTS OUTPUTS Conceptual Graphs OBSERVER ACTORS SENSE THE STATE OF THE KNOWLEDGE IN THE CONCEPTUAL GRAPHS AND TRANSLATE THAT INTO THE SPECIFIC OUTPUT FORMAT NEEDED BY THE USER

17. Human Factors Architecture Development Model Validation Human Factors Assessment Human Factors Modeling Cognitive Psychological Behavior Architecture Characteristics Moderators Moderators Modifiable Psychological Behavior Cognitive Synthetic Psychological Cognitive Assessment Moderator Architecture Assessment Architecture Battlespace Tools Identification Modification Human Cognitive Architecture Psychological Behavior Moderators : Commander Moderators : Characteristics : Traits Structural Traits : Control Architecture topology Performance Under Pressure Introversion Extraversion ( Weights on intermodule links ) Intellectual Expressiveness Long - term memory Concentration Style (War)game Self-Esteem ) or Synthetic ( Content & structure States Behavioral Control Anxiety Physical Orientation Environment Frustration Processing Human Obsessiveness Cognitive Module parameters Commander Cognitive Focus ( Capacity and speed ) Attention capacity / Distractibility / speed Flexiblity Inferencing speed & biases WM capacity / speed Internal External ( Cue selection & delays ) Skill Awareness

18. Internal ProcessesAnalogy: The Heart Beat • Atria pump blood to ventricles, which contract • Nonstop contractions are driven by the heart's electrical system Internal Process: Synchronous or Asynchronous – Intrinsic Capabilities

19. External ProcessesAnalogy: Pacemaker • External process monitors and interacts with an object (i.e., a pacemaker monitors the heart’s rhythm) • The electric current makes the heart beat within a certain range External Process: Synchronous or Asynchronous – Monitor and Control

20. Internal Events Analogy: Heart Attack • Internal occurrence without pre-established time scale • Certain factors cause the occurrence. Blood flow is restricted, or the nerve system, which controls the heart, malfunctions Internal Occurrence: Irregular Time Scale – Intrinsic Capabilities

21. External EventsAnalogy: Defibrillation • External event changes a passive object’s state (i.e., a defibrillator is used for resuscitation) • External electrical shock is applied to the heart • Basic representation method External Occurrence: Irregular Time Scale – Monitor and Control

22. Conceptual Graphs

23. Basic CG Formation Rules

24. CG Execution

25. CG Execution Example

26. IT Competitor Comparison

27. LSI Competitor Comparison