Interactive Evolutionary Computation: A Framework and Applications

1. Interactive Evolutionary Computation:A Framework and Applications November 10, 2009 Sung-Bae Cho Dept. of Computer Science Yonsei University

2. Agenda • Overview • Methodology • IGA • Knowledge-based encoding • Partial evaluation based on clustering • Direct manipulation of evolution with NK-landscape model • Applications • Media retrieval • Media design • Concluding remarks

3. Overview Interactive Computational Intelligence Interaction/Evolution Human Computer Interface User Modeling (Efficiency) (Emotion) AI (Soft Computing) FL GA NN

4. Overview Interactive Evolutionary Computation (IEC) • Difficulty of optimization problems in interactive system • Outputs must be subjectively evaluated (graphics or music) • Definition • Evolutionary computation that optimizes systems based on subjective human evaluation • Fitness function is replaced by a human user My goal is … Target system f(p1,p2,…,pn) System output Subjective evaluation Interactive EC From Takagi (2001)

5. Overview Inherent Problems in IEC • Human fatigue problem • Common to all human-machine interaction systems • IEC • Acceleration of EC convergence with small population size and a few generation numbers • Usually within 10 or 20 search generations • Limitation of the individuals simultaneously displayed on a screen • Limitation of the human capacity to memorize the time-sequentially displayed individuals • Requirement to minimize human fatigue I’m tired

6. Overview Direct Manipulation Media Retrieval & Design Media Retrieval & Design Partial Evaluation IEC IEC Domain Knowledge Proposed Conventional Proposed Framework

7. Methodology Interactive Genetic Algorithm

8. Methodology Knowledge-based Encoding New Search Space Removing impractical solutions Effective encoding? Using partially known information about the final solution Domain Knowledge Focusing on a specific search space Encoding in a modular manner Search Space

9. Methodology Partial Evaluation based on Clustering Issues  Clustering algorithm selection, Indirect fitness allocation strategy

10. Methodology It is very similar to the final solution but a bit different. Direct Manipulation of Evolution IGA Direct Manipulation Proof of usefulness of the direct manipulation  NK-landscape

11. Applications Overview Humanized Media Retrieval & Design Media Retrieval Media Design Image Retrieval Video Retrieval Music Retrieval Fashion Design Flower Design Intrusion Pattern Design

12. Applications: Image Retrieval Content-based Image Retrieval • Keyword-based method • Much time and labor to construct indexes in a large databases • Performance decrease when index constructor and user have different point of view • Inherent difficulty to describe image as a keyword • Content-based method • Image contents as a set of features extracted from image • Specifying queries using the features

13. Applications: Image Retrieval Motivation • Systems for content-based image retrieval • QBIC (IBM, 1993) • QVE (Hirata and Kato, 1993) • Chabot (Berkeley, 1994) • Needs for image retrieval based on human intuition • Difficult to get perfect expression by queries • Lack of expression capability

14. Applications: Image Retrieval System Structure

15. Applications: Image Retrieval Original image Transformed image Chromosome Chromosome Structure R G B 0 1 2 3 49 50

16. Applications: Image Retrieval Initial population The 8th population Convergence Test • The case of gloomy image retrieval

17. Applications: Image Retrieval 95% Confidence interval 99% Subjective Test by Sheffe’s Method

18. Applications: Video Retrieval Motivation • Evolution of computing technology (Huge computing storage, networking) • Necessary for management of video data transfer, processing and retrieval • Change of view point • Query by keyword  Query by content  Difficult to represent human’s semantic information • Query by semantics • Emotion-based retrieval

19. Applications: Video Retrieval Hierarchical Structure of Video

20. Applications: Video Retrieval System Architecture

21. Applications: Video Retrieval Chromosome Structure

22. Applications: Video Retrieval System Interface

23. Applications: Video Retrieval User’s Satisfaction

24. Applications: Music Retrieval Emotional Music Retrieval • Music information retrieval  Immature field • Query by singing • Query by content • Uploading pre-recording humming via web browser • Typing the string to represent the melody contour • Preprocessing • Query by humming  transforms user’s humming into symbolized representation • Conventional music retrieval system • Fast and effective extraction of musical patterns and transformation of user’s humming into systematic notes • Problem : User cannot remember some parts of melody

25. Applications: Music Retrieval Power Spectrum Analysis a : classic b : jazz c : rock d : news channel

26. Applications: Music Retrieval System Architecture

27. Applications: Music Retrieval Chromosome Structure

28. Applications: Music Retrieval Convergence Test (1)

29. Applications: Music Retrieval Convergence Test (2)

30. Applications : Fashion Design Change of Consumer Economy Before the Industrial Revolution : Customers have few choices on buying their clothes Manufacturer Oriented After the Industrial Revolution : Customers can make their choices with very large variety Near Future : Customers can order and get clothes of their favorite design Consumer Oriented

31. Applications : Fashion Design Need for Interactive System • Almost all consumers are non-professional at design • To make designers contact all consumers is not effective • Need for the design system that can be used by non-professionals

32. Applications : Fashion Design Fashion Design • Definition • To make a choice within various styles that clothes can take • Three shape parts of fashion design • Silhouette • Detail • Trimming

33. Applications : Fashion Design System Architecture

34. Applications : Fashion Design Knowledge-based Encoding A B C D E F Total 23 bits E : Skirt and waistline style(9) F : Color(8) B : Color(8) A : Neck and body style(34) … … D : Color(8) C : Arm and sleeve style(11) Search space size =34*8*11*8*9*8 =1,880,064 …

35. Applications : Fashion Design Direct Manipulation Interface

36. Applications : Fashion Design Fitness Changes for Encoding Schemes

37. Applications : Fashion Design Hypercube Analysis Shortest Path Using DM Method One-Mutant Neighbors using Genetic Operator

38. Applications : Flower Design Motivation • Proposed Method • Representation of knowledge-based genotype using the structure of real flower • Automatic generation of creatures • Process • Directed graph: Define the genotype of flower structure • IGA: Evaluation of created individuals and automatic creation • Math Engine: Representation of phenotype • Objective • Creation of character morphology similar to real flower • Find optimal solution in small solution space using knowledge-based genotype representation such as directed graph • Automatic creation of character using IGA

39. Applications : Flower Design Related Works • Tree, flower and feather modeling • L-System • Box-based artificial characters (Karl Sims) evolution • Graph model representation • Golem • Robotic form and controller evolution • Graph-based data structure

40. Applications : Flower Design Overall Procedure

41. Applications : Flower Design Chromosome Structure

42. Applications : Flower Design Convergence Test

43. Applications : Flower Design Subjective Test by Sheffe’s Method

44. Applications : Intrusion Pattern Design Motivation (1) • Various intrusion detection systems • For evaluating, IDS uses known intrusions • Possibility of having different patterns within identical intrusion type • Difficult to detect transformed intrusions • Problem • For better evaluation, intrusion patterns are needed more • Difficult to generate all possible proper intrusion patterns manually • Automatic generation is needed

45. Applications : Intrusion Pattern Design Motivation (2) • Evolutionary pattern generation • Needs a variety of intrusion patterns • It is possible to generate new patterns by combining primitives • Generation of transformed intrusion patterns using simple genetic algorithm • IGA-based intrusion pattern generation • Difficult to estimate the fitness of patterns automatically • Using interactive genetic algorithm for the evaluation of patterns

46. Applications : Intrusion Pattern Design Method • Intrusive behavior can be conducted in 5 steps • Proposed by DARPA • Possible path of U2R attacks

47. Applications : Intrusion Pattern Design Known Intrusion Patterns IGA Intrusion Patterns GA 000100..... Initial Population Fitness Evaluation 001100..... Exploit Code Generate Audit Data System Architecture

48. Concluding Remarks • Humanized CI framework using IEC • Knowledge-based encoding • Partial evaluation based on clustering • Direct manipulation of evolution with NK-landscape • Applications in media retrieval & design • Image, video and music retrievals • Fashion, flower and intrusion pattern designs • Contribution of humanized CI to engineering fields  Developing Emotional HC Interface by Evolving models through Interaction with Humans