Probabilistic State Machines to describe emotions

1. Probabilistic State Machines to describe emotions Human speaks to robot “you are beautiful” / ”Thanks for a compliment” P=1 Robot speaks to human (will be in italic in next slides) Happy state “you are blonde!” / ”I am not an idiot” P=0.3 “you are blonde!” / Do you suggest I am an idiot?” Unhappy state P=0.7 Ironic state

2. Facial Behaviors of Maria Do I look like younger than twenty three? Maria asks: Response from a human: • “no” • “no” • “yes” 0.7 0.3 In word spotting mode human responses are short Maria smiles Maria frowns

3. Probabilistic Grammars for performances Speak ”Professor Perky”, blinks eyes twice P=0.1 Speak ”Professor Perky” Where? P=0.3 Who? P=0.5 P=0.5 P=0.5 Speak “in some location”, smiles broadly Speak “In the classroom”, shakes head Speak ”Doctor Lee” What? P=0.1 P=0.1 P=0.1 Speak “Was singing and dancing” P=0.1 Speak “Was drinking wine” A complete behavior of a robot described as a probabilistic (quantum entangled) machine ….

4. Human-controlled modes of dialog/interaction Human teaches initialization “Thanks, I have a lesson” “Hello Maria” “Lesson finished” Robot performs Robot asks “Question” “Stop performance” “Questioning finished” “Command finished” This slide shows only human speech controls, no robot responses showing its state “Thanks, I have a question” “Thanks, I have a command” Human asks Human commands

5. The next slides will illustrate the following robot behaviors: • Robot Initiated Conversation. • Human-Initiated Conversation. • Human asking. • Human teaching. • Human verifying what robot learned. Other robot behaviors: • Robot Performing (text, music, movements). • Game Playing.

6. Robot-Receptionist Initiated Conversation Human Robot What can I do for you? Robot asks This represents operation mode

7. Robot-Receptionist Initiated Conversation Human Robot What can I do for you? I would like to order a table for two Robot asks

8. Robot-Receptionist Initiated Conversation Human Robot Smoking or non-smoking? Robot asks

9. Robot-Receptionist Initiated Conversation Human Robot Smoking or non-smoking? I do not understand Robot asks

10. Robot-Receptionist Initiated Conversation Human Robot Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. Robot asks

11. Robot-Receptionist Initiated Conversation Human Robot Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. A table near the terrace, please Robot asks

12. Human-Initiated Conversation Human Robot Hello Maria initialization Robot asks

13. Human-Initiated Conversation Human Robot Hello Maria What can I do for you? Robot asks

14. Human-Asking Human Robot Question Question Robot asks Human asks

15. Human-Asking Human Robot Yes, you ask a question. Question Human asks

16. Human-Asking Human Robot Yes, you ask a question. What book wrote Lee? Human asks

17. Human-Asking Human Robot I have no sure information. What book wrote Lee? Human asks

18. Human-Asking Human Robot I have no sure information. Try to guess. Human asks

19. Human-Asking Human Robot Lee wrote book “Flowers”. Try to guess. Human asks

20. Human-Asking Human Robot Lee wrote book “Flowers”. This is not true. Human asks

21. Human ends questioning Human-Teaching Human Robot Questioning finished Human asks Human teaches “Questioning finished” Robot asks Thanks, I have a lesson

22. Robot enters asking mode Human-Teaching Human Robot What can I do for you? Questioning finished Human asks Human teaches “Questioning finished” Robot asks Thanks, I have a lesson

23. Human starts teaching Human-Teaching Human Robot What can I do for you? Thanks, I have a lesson Human asks Human teaches “Questioning finished” Robot asks Thanks, I have a lesson

24. Human-Teaching Human Robot Thanks, I have a lesson Yes Human teaches

25. Human-Teaching Human Robot I give you question-answer pattern Yes Human teaches

26. Human-Teaching Human Robot Question pattern: What book Smith wrote? Yes Human teaches

27. Human-Teaching Human Robot Answer pattern: Smith wrote book “Automata Theory” Yes Human teaches

28. Human-Teaching Human Robot Checking question: What book wrote Smith? Yes Human teaches

29. Human-Teaching Human Robot Checking question: What book wrote Smith? Smith wrote book “Automata Theory” Human teaches

30. Human-Teaching Human Robot I give you question-answer pattern Yes Human teaches

31. Human-Teaching Human Robot Question pattern: Where is room of Lee? Yes Human teaches

32. Human-Teaching Human Robot Answer pattern: Lee is in room 332 Yes Human teaches

33. Human-Checking what robot learned Human Robot Lesson finished Human teaches “Lesson finished” Question Robot asks Human asks

34. Human-Checking what robot learned Human Robot Lesson finished What can I do for you? Human teaches “Lesson finished” Question Robot asks Human asks

35. Human-Checking what robot learned Human Robot Question What can I do for you? Human teaches “Lesson finished” Question Robot asks Human asks

36. Human-Asking Human Robot Yes, you ask a question. Question Human teaches “Lesson finished” Question Robot asks Human asks

37. Human-Asking Human Robot Yes, you ask a question. What book wrote Lee? Human asks

38. Human-Asking Human Robot I have no sure information. What book wrote Lee? Human asks

39. Human-Asking Human Robot I have no sure information. Try to guess. Human asks

40. Human-Asking Human Robot Lee wrote book “Automata Theory” Try to guess. Observe that robot found similarity between Smith and Lee and generalized (incorrectly) Human asks

41. How we linked Behavior, Dialog and Learning • The dialog/behavior has the following components: • (1) Eliza-like natural language dialogs based on pattern matching and limited parsing. • Commercial products like Memoni, Dog.Com, Heart, Alice, and Doctor all use this technology, very successfully – for instance Alice program won the 2001 Turing competition. • This is a “conversational” part of the robot brain, based on pattern-matching, parsing and black-board principles. • It is also a kind of “operating system” of the robot, which supervises other subroutines.

42. Behavior, Dialog and Learning • (2) Subroutines with logical data base and natural language parsing (CHAT). • This is the logical part of the brain used to find connections between places, timings and all kind of logical and relational reasonings, such as answering questions about Japanese geography. • (3) Use of generalization and analogy in dialog on many levels. • Random and intentional linking of spoken language, sound effects and facial gestures. • Use of Constructive Induction approach to help generalization, analogy reasoning and probabilistic generations in verbal and non-verbal dialog, like learning when to smile or turn the head off the partner.

43. Behavior, Dialog and Learning • (4) Model of the robot, model of the user, scenario of the situation, history of the dialog, all used in the conversation. • (5) Use of word spotting in speech recognition rather than single word or continuous speech recognition. • (6) Avoidance of “I do not know”, “I do not understand” answers from the robot. • Our robot will have always something to say, in the worst case, over-generalized, with not valid analogies or even nonsensical and random.

46. Generalization of the Ashenhurst-Curtis decomposition model

47. This kind of tables known from Rough Sets, Decision Trees, etc Data Mining

48. Decomposition is hierarchical At every step many decompositions exist

49. Constructive Induction: Technical Details • U. Wong and M. Perkowski, A New Approach to Robot’s Imitation of Behaviors by Decomposition of Multiple-Valued Relations, Proc. 5th Intern. Workshop on Boolean Problems, Freiberg, Germany, Sept. 19-20, 2002, pp. 265-270. • A. Mishchenko, B. Steinbach and M. Perkowski, An Algorithm for Bi-Decomposition of Logic Functions, Proc. DAC 2001, June 18-22, Las Vegas, pp. 103-108. • A. Mishchenko, B. Steinbach and M. Perkowski, Bi-Decomposition of Multi-Valued Relations, Proc. 10th IWLS, pp. 35-40, Granlibakken, CA, June 12-15, 2001. IEEE Computer Society and ACM SIGDA.

50. Constructive Induction • Decision Trees, Ashenhurst/Curtis hierarchical decomposition and Bi-Decomposition algorithms are used in our software • These methods create our subset of MVSIS system developed under Prof. Robert Brayton at University of California at Berkeley [2]. • The entire MVSIS system can be also used. • The system generates robot’s behaviors (C program codes) from examples given by the users. • This method is used for embedded system design, but we use it specifically for robot interaction.