Symbol-Anchoring in Cassie

1. Symbol-Anchoring in Cassie Stuart C. Shapiro and Haythem O. Ismail Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo {shapiro | hismail}@cse.buffalo.edu S.C. Shapiro

2. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

3. Embodied Cassie • A computational cognitive agent • Embodied in hardware • or Software-Simulated • Based on SNePS and GLAIR. S.C. Shapiro

4. Cassie, the FEVAHR S.C. Shapiro

5. FEVAHR/Cassie in the Lab S.C. Shapiro

6. SNePS • Knowledge Representation and Reasoning • Intensional Representation • Propositions as Terms • SNIP: SNePS Inference Package • Specialized connectives and quantifiers • SNeBR: SNePS Belief Revision • SNeRE: SNePS Rational Engine • Interface Languages • SNePSUL: Lisp-Like • SNePSLOG: Logic-Like • GATN for Fragments of English. S.C. Shapiro

7. Interaction with Cassie (Current) Set of Beliefs [SNePS] English (Statement, Question, Command) Reasoning Clarification Dialogue Looking in World GATN Parser (Updated) Set of Beliefs [SNePS] (New Belief) [SNePS] Answer [SNIP] Actions [SNeRE] GATN Generator Reasoning English sentence expressing new belief answering question reporting actions S.C. Shapiro

8. Entities, Terms, Symbols, Objects • Cassie’s mental entity: a person named Stu • SNePS term: B4 • Object in world: S.C. Shapiro

9. GLAIR Architecture Grounded Layered Architecture with Integrated Reasoning Knowledge Level SNePS Perceptuo-Motor Level NL Sensory-Actuator Level Vision Sonar Proprioception Motion S.C. Shapiro

10. Alignment KL SNePS term PML structure Body (PML/SAL) World Object/Phenomenon Action S.C. Shapiro

11. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

12. World Objectsto Feature Vectors PML/SAL World <Height, Width, Texture, . . . > S.C. Shapiro

13. Feature Vectorsto KL Terms KL PML/SAL ProperName(B4, Stu) <Height, Width, Texture, . . . > Alignment S.C. Shapiro

14. Incomplete Feature Vectors KL PML/SAL Height(B4, B12) <Height, nil, nil, . . . > S.C. Shapiro

15. Unifying PML-Descriptions PML/SAL KL <x, nil, nil, ... > B20 <nil, y, nil, ... > B30 Prop Isa <nil, nil, z, ... > B31 Prop <x, y, z, ... > B6 S.C. Shapiro

16. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

17. Deictic & Modality Registers KL Terms denoting PML Cassie Addressee Current time Current state(s) of looking at x . . . I You Now Vision . . . S.C. Shapiro

18. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

19. Primitive Actions Alignedwith PML/SAL Functions KL PML SAL World Find(B4) Find(B4) <... > Modality Registers S.C. Shapiro

20. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

21. Aligning NOW using MTF Maximal Temporal Frame based on *NOW KL PML NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states S.C. Shapiro

22. Moving NOW with MTF KL NOW PML Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers. S.C. Shapiro

23. ! duration time before after ! Providing a Feel for Time q t1 t2 KL PML hom COUNT n NOW 0 S.C. Shapiro

24. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

25. Aligning Lexemes/NL KL PML/SAL World ProperName(B4, Stu) npr ctgy “Stu” Isa(B4, person) n ctgy “person” sing num Grammar NL S.C. Shapiro

26. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

27. Acting 1 S.C. Shapiro

28. Acting 2 Follow a red robot. I found a red robot. I am looking at a red robot. S.C. Shapiro

29. Acting 3 Follow a red robot. I found a red robot. I am looking at a red robot. I went to a red robot. I am near a red robot. I am following a red robot. S.C. Shapiro

30. Use of Indexicals 1 Come here. I found you. I am looking at you. S.C. Shapiro

31. Use of Indexicals 2 Come here. I found you. I am looking at you. I came to you, Stu. I am near you. S.C. Shapiro

32. Use of Indexicals 3 Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. S.C. Shapiro

33. Use of Indexicals 4 Come here. I found you. I am looking at you. S.C. Shapiro

34. Use of Indexicals 5 Come here. I found you. I am looking at you. I came to you. I am near you. S.C. Shapiro

35. Use of Indexicals 6 Who am I? Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? I talked to Stu and I am talking to you. S.C. Shapiro

36. Outline • Introduction • Perceivable entities and properties • Attentional Structures • Actions • Time • Language • Examples • Summary S.C. Shapiro

37. Summary • KL terms denote mental entities. • KL terms aligned to PML structures. • PML variables/registers contain KL terms. • PML variables/registers/structures grounded in world via sensors & effectors. • PML attentional structures anchor symbols by changing only when motivated. S.C. Shapiro

38. For More Information • Personnel • Manual • Tutorial • Bibliography • ftp’able SNePS source code • etc. • http://www.cse.buffalo.edu/sneps/ S.C. Shapiro