V-Lab  – A Soft Computing Based Virtual Laboratory for Intelligent Agents

1. V-Lab– A Soft Computing Based Virtual Laboratory for Intelligent Agents Mo Jamshidi Electrical and Computer Engineering Department and Autonomous Control Engineering (ACE) Center University of New Mexico, Albuquerque Advisor, NASA HQ, US DOE HQ and US Air Force Research Lab. http://ace.unm.edu jamshidi@eece.unm.edu & moj@cybermesa.com December 15, 2003 – UC Berkeley

2. OUTLINE What is V-Lab? NASA Mission Applications V-Lab Architecture and Discrete-Event Systems Specifications (DEVS) Intelligent DEVS Examples Conclusions Demo Simulations & Movies

3. V-Lab®is based on … … a NASA Research Announcement (NRA), Cross-Enterprise Grant with NASA Ames Research Center and a joint effort between UNM ACE Center and NASA, in cooperation with the Arizona Integrate Modeling and Simulation Center (U Arizona – Dr. Bernie Zeigler and ASU - Dr. Hessam Sarjoughian). Dr. Edward Tunstel, Jr. of JPL and Dr. Chuck Jorgenson of NASA Ames were two of the 4 Co-PIs on the grant.

4. Titan Blimp Solar Montgolfiere Jupiter Balloon MER V-Lab®’s potential role for NASAMission Context for Mobility Systems R&D Airborne Systems Mars Airplane 10 km Science Outpost Inflatables Surface Systems 1 km Sojourner Nanorovers 100 m Mars Smart Lander/Rover… MSR Sub-surface Systems 10 m Sub-surface Explorer 1 m Cryobot Hydrobot Deep Drills 0.1 m US Drill/ Corer Mini-corer State of the Art (SOA) SOA + 5 years SOA + 10 years

5. Entitiesof Modeling &Simulation Experimental Frame (Communication) Simulate Real world (Ant Colony) Model

6. Entitiesof Modeling &Simulation • Discrete Time Simulation: • Continuous signal is sampled at different time intervals. State change at each of these intervals is examined. • Discrete Event simulation: • State change of the system is examined only at arrival of an event, not at equally spaced time instances. Time T0 T1 T2 T3

7. V-Lab®… Environment Layers • V-Lab® Environment Layers • Hardware Networking • Networking foundation for inter-machine communication • Middleware • Software environment for inter-process communication • IDEVS • Library of tools for soft computing in DEVS formalism • V-Lab® • Organizational structure for DEVS objects. • Management objects to control time and message flow in multi-agent systems V-Lab® I-DEVS MIDDLEWARE PHYSICAL NETWORK

8. Inside Layered Architecture SimMan Atomic Model Coupled Model middleware

9. Allows for virtual communication between objects on a Client/Server basis Provides ORB for network communication Provides language to create Stubs/Skeletons Allows for processes on multiple platforms to communicate Server Module Client Module Virtual Communication Skeleton Stub ORB ORB Network CORBA

10. DEVS Introduction • Event: “The representation of an instantaneous change in some part of a system”. Characterized by: • A value: can be a number, a vector, a word, or in general, an element of a given set. • The occurrence time.

11. DEVS(Discrete Event System Specification) DEVS Atomic =(X, Y, S, ext, int, , ta) • X:is the set of inputs, • Y:is the set of outputs, • S: is the set of state values, • int :is the internal state transition function, •  : is the output function, • ta : is the time advance function. • ext :is the external state transition function,

12. t0 t1 t2 s2 e s1 int ext int s3 ta(s0) t1 ta(s2)   t0 t2 Behavior of DEVS Models Atomic DEVS=(X, Y, S, ext, int, , ta) X S s0 ta(s1) Y

13. “Idle” Done Fix On “Broken-Down” “Working” Brk Discrete Event System • A discrete event system is a dynamic system whose behavior is guided by the occurrence of discrete events. • Example: a 3-state machine

14. Simulation Design Qualities • Layered Architecture • Breaks up the entire simulation into layers, each with a distinct purpose and function • Object Modularity • Functional components of the individual layers should be broken up into distinct objects to allow for: • Maintainability – Changing a single object without modifying entire simulation. • Reuse – Distinct objects can be reused in different simulations • Object Hierarchy – Provide a structure for the composition and communication of objects • Our Solution – V-Lab®

15. DEVS • Provides C++ and Java classes for object design • Hierarchical structure of objects • Two Types of Objects • Coupled (A-BC, B-C) • Contains other objects • Atomic (A, B, C) • Basic functionality to be used by coupled objects A-BC A B-C B C Hierarchal tree A-BC B-C A B C Coupling Relation

16. DEVSexecution • Create Models • Determine imminent models • Imminent models send messages to output ports • Imminent models call Internal Transition function • Other models receive external messages • Other models call External Transition function • Repeat until no models can be imminent

17. Uses the DEVS structure for creating objects Organizes objects into 6 different categories SimEnv/SimMan Terrain Models Agent Models Physics Models Control Models Dynamic Models Provides objects for the time and message flow of simulation SimEnv Terrain Physics Agents Control SimMan Dynamic V-Lab Hierarchical Tree

18. SimEnv Highest leveled coupled model Instatiates all other models Houses all other models Couples all other models together SimMan Message liaison providing indirection between all other models Controls flow of time for the simulation SimManandSimEnv SimEnv Agent Control Physics in/out in/out in/out in out SimMan in out in/out in/out Dynamic Terrain Coupling Relation

19. Simulation execution is iteration through a 5 phase cycle Phase 1 Check for termination conditions Phase 2 Update Agent Sensors Phase 3 Control Algorithms update Agent Actuators Phase 4 Wait for all actuators to change state Phase 5 Dynamic models update agents and environment V-Lab®Cycle

20. IDEVS …Intelligent DEVS DEVS Coupled Models Discrete Events Atomic Models Soft Computing

21. Fuzzy Logic Neural Network Soft Computing DEVS I-DEVS Stochastic Learning Automaton Genetic Algorithms IDEVS …Intelligent DEVS 2 Other tools

22. Fuzzy Logic Controllers Inference Engine Numerical Values Fuzzifier Defuzzifier Numerical Values Union of Decisions Membership Functions Sensor Variables Membership Functions Fuzzy (IFTHEN) Rules Fuzzy Variables Linguistic Variables

23. Fuzzifier DOF X (real number) Method of Defuzzification Membership Function Defuzzifier Y1 AND/OR Y2 DOFs DOF Defuzzified output (real number) Output Block … … Y (Fuzzy set) DOF Yn Fuzzy DEVS DEVS primitives to compose a fuzzy rule:

24. A x C z OR B y CMF AMF μ μ X Y X Y μ X Y Fuzzy-DEVS Object: Implement various fuzzy functions and operations within a DEVS Java environment. Example – fuzzy inferencing A DEVS model for a typical fuzzy rule: IF x is A OR y is B THEN z is C

25. Fuzzy-DEVS Controlof Inverted Pendulum

26. POS ZERO NEG a) Membership functions for fuzzy-DEVS b) The fuzzy-DEVS controller c) The close-loop systemd) The output of the system () Fuzzy-DEVS, Cont’d. Example of using fuzzy-DEVS to control an inverted pendulum

27. Stop or Continue? Generate Initial Population Get Fitness & Selection Intermediate Population Reproduce New Population Final Solution GA-DEVS Input Conditions Basic cycle of geneticalgorithms

28. GA-DEVS, Cont’d. GA-DEVS implementation inside V-Lab®

29. GA-DEVS, Cont’d. 2 A simulation window of GA-DEVS

30. GA-DEVS, Cont’d. 3 Average and maximum fitness versus generation for an example problem

31. SLA-DEVS Stochastic Learning Automaton – A statistical Learning approach to learning … Schematic of SLA inside the DEVS Environment

32. Stochastic Learning Automaton(SLA) as a component ofV-Lab® • Statistical learning • Model-free • No specific knowledge of objective function • Blends well in soft computing • Learning is defined as any permanent change in behavior as a result of past experience, and a learning system should therefore have the ability to improve its behavior with time, toward a final goal. • In a purely mathematical context, the goal of learning system is the optimization of a function not known explicitly.

33. What is a Stochastic Learning Automata? • SLA is a sequential machine. • Given a finite number of actions that can be performed in a random environment, when a specific action is taken place the environment provides a random response which is either favorable or unfavorable. • The objective in the design of the automaton is to determine how the choice of the action at any stage should be guided by past actions and responses.

34. Learning Automata Single Teacher

35. Stochastic Learning Automata– Reinforcement Learning Stochastic Learning Automaton Update Action Probabilities s(n) y(n) G(.) F(w(n),s(n))

36. Simulation, Cont’d … 2 Agents (a) (b) (c) (d)

37. Simulation, Cont’d … 2 Agents (f) (e) (h) (g)

38. NN-DEVS Neural Networks – A neural-based approach to learning … A perceptron structure inside IDEVS

39. Back PropagationNN-DEVS A 4 layer BPNN is now be implemented in DEVS

40. V-Lab® Simulations

41. ROBOT IR Sensor 1 in out IR Sensor 2 . . . RobotDyn IR Sensor n Compass Controller Position Sensor V-Lab®: Robot Model

42. d Obstacle r ( Sensor Max Range) s IR-Sensor (xs, ys) V-Lab®: Terrain Model

43. Mass 1 Mass 2 in out . . . Mass n V-Lab®: Dynamics Model

44. V-Lab®

45. SimEnv Plot Model Terrain Model Rover Dynamics Sensors SimMan Controller EXAMPLE1 Robotic Colony Block Diagram of Example 1

46. EXAMPLE1 SIMULATION – Rover Dynamics dx/dt=vcos dy/dt=vcsin d/dt=w Rover Atomic Model

47. a) Distance to objective) Angle error b) Angle error c) Forward velocity) Angular velocity. d) Angular velocity. EXAMPLE1 SIMULATION – Fuzzy Controller Rule1: IF distance is ‘Near’, THEN forward velocity is ‘Slow’. Rule2: IF distance is ‘Far’, THEN forward velocity is ‘Speedy’. Rule3: IF angle is ‘Right’, THEN angular velocity is ‘TurnRight’; forward velocity is ‘Slow’. etc.

48. Motion Control of Mobile Robots • Point-to-point motion (goal searching): The robot is assigned to reach a desired goal configuration starting from a given initial configuration, while avoiding collision with obstacles. This task is sometimes called Path planning in the literature • Path following: The robot must reach and follow a geometric path in the Cartesian space starting from a given initial configuration (on or off the path). • Trajectory tracking: The robot must reach and follow a trajectory in the Cartesian space (i.e., a geometric path with an associated timing law) starting from a given initial configuration. .

49. Desired Position Desired Velocity Torque Volt Velocity Position Current Velocity Position Robot Motion Control

50. Path Planning Goal Configuration Initial Configuration