Case Injected Genetic Algorithms for Affordable Human Modeling Start Date: 11/15/02

1. Case Injected Genetic Algorithms for Affordable Human ModelingStart Date: 11/15/02 Sushil J. Louis University of Nevada, Reno John McDonnell SPAWAR San Diego

2. Case Injected Genetic AlgoRithms (CIGARs) combine genetic algorithms and case-based reasoning to address three problems • Affordability: The system will be used for both decision support and training. Dual use saves money and familiarizes trainees with battlefield systems • Human Modeling: CIGAR uses cases captured from humans during decision support, war-gaming, and training to bias genetic algorithm search toward human solutions • Quality of opponent: CIGAR automatically acquires knowledge (generates cases) by playing against itself. This bootstrapping leads to better quality opponents and reduces knowledge acquisition cost

3. Case Injected Genetic AlgoRithms (CIGAR)s combine Genetic Algorithm (GA) search with Case-Based Memory GAs adapt solutions (cases) acquired from previously attempted problems to solve subsequent problems We think of CIGAR as an optimization engine that acquires cases from problem solving or from humans and learns to increase performance with experience Technical Objective:Prototype and validate CIGAR techniques for more robust, more affordable human behavior modeling Problem 1 Problem 2 Problem 3 Human CIGAR

4. Training: Provide trainee with quality opponent (strike planner) Training: Provide trainee with quality opponent (target/threat configurator) Decision Support: Assist decision makerin configuration Decision Support: Assist decision maker in time-critical strike Cigar achieves more affordable decision support and training for naval applications The Real-time Executive Decision Support (REDS) effort at SPAWAR will use CIGAR as an optimization engine in strike force weapon-target pairing. Platforms Targets and Threats Transition objective: Generate multiple weapon-target pairing options in less than 4 minutes for 20 weapon-target pairs. Include SEAD support and METOC information

5. CIGAR is affordable The system uses the same graphical user interface for decision support and for training. Decision makers use the interface to specify solutions to decision problems – these solutions are cases and are acquired as a by-product of operational use. Trainees use the same interface for training/wargaming. This dual use translates into significant cost savings and acquires domain knowledge Motto: Fight as you train, train as you fight Fighting Training CIGAR

6. CIGAR produces high quality solutions System use or operation by humans acquires cases representing domain knowledge. CIGAR also acquires cases as it solves problems generated by a problem generator. This offline knowledge acquisition will lead to better performance for training and for decision support. Replacing the problem generator with CIGAR, we can evolve quality opponents CIGAR Problem Generator Problem 1 Problem 2 Problem 3 CIGAR

7. Candidate solutions Cases Case-base Cases Candidate solutions CIGAR acquires knowledge during problem solving • Periodically save members of the GA’s population to the case-base • A member of the population is a candidate solution to the problem • Periodically inject appropriate cases into the GA’s population replacing low-fitness members Save best individual CBR module Preprocessor Genetic Algorithm Preprocessor Inject closest to the best

8. How does CIGAR operate ? • Which cases do we inject? • Inject cases that are closest to the current best member of the population. Genetic algorithms usually use binary encodings. For these encodings, our distance metric is therefore Hamming Distance – the number of differing bits. • GA theory points to other injection strategies • Probabilistic version: The probability of injection of a case in the case base is inversely proportional to distance from the current best member of the population relative to the distances of other cases. • How often should we inject cases? • Takeover time – number of generations needed for an individual to take over the population. P(Casei) = (l – di)/∑(l – dj) Hamming distance from best member Chromosome length Sum over all cases

9. Expected behavior of a learning system Learning system/CIGAR However, we need to deal with few cases captured from humans and obtain (1) human-like and (2) high-quality solutions Quality No learning Number of problems attempted No learning Time This performance is with (1) a simple problem generator and (2) a case base that grows large Learning system/CIGAR Number of problems attempted Expected behavior versus actual behavior CIGAR behavior on 50 design problems Avg. best fitness found within a max of 30 generations Number of problems attempted Avg. time taken to find best fitness Number of problems attempted RIGA = Randomly Initialized Genetic Algorithm

10. Can injecting a fewcases captured from humans result in (1) high-quality and (2) timely solutions? CIGAR solutions are similar to injected cases • The graph below displays hamming distance as a function of chromosome location. • At a number of locations, CIGAR solutions are more similar to each other • Other analysis shows that CIGAR solutions are descendants of injected cases  When injected cases come from humans, CIGAR will tend to produce solutions similar to humans Avg. hamming distance Chromosome position (locus)

11. Note that in this case CIGAR takes decreasing time but there is little difference in quality ? Performance on weapon-target pairing Objective function to maximize effectiveness value & risk Given allocation X, U(X) depends on pilot proficiency with weapon & weapon’s effectiveness on target. V(X) describes marginal effect of using multiple weapons Y(X) depends on routing, SEAD, METOC…

12. We have built a foundation for delivering on research objectives. Year 1 deliverables: • Deliverables related to technical objective • Affordability • Deploy a prototype GUI for weapon-target pairing support on the web • Demonstrate dual use for decision support and training/war-gaming • Human Modeling • A set of tools for case-base analysis • Analysis of empirical results from injecting cases acquired from a human expert. Techniques for dealing with few human cases • Techniques for combining CIGAR and human cases • Deliverables related to transitioning objective • Provide an optimization engine that integrates into the REDS-KSA architecture

13. Years 2 and 3 • Related to the technical objective • Prototype and deploy a CIGAR system as the red-force against weapon-target pairing • Demonstrate competent red-force scenario generation against weapon-target pairing • Demonstrate techniques for co-evolving blue-red forces • Test and validate approaches to combining human generated cases with automatically acquired cases • Related to the transitioning objective • Demonstrate < 4 minutes for 20 weapon-target pairs with SEAD support/routing and METOC data • Other military applications

14. System Architecture Simulation Comm. Hub Physics Gfx Defense CIGAR Defense Planning Decision Support Battle Authoring Offense CIGAR Offense Planning Decision Support GUIs

15. Questions? Tools being developed http://gaslab.cs.unr.edu

17. Solution distribution