Partner Robots for Mental Health Care - Japanese Challenge toward Practical Robots -

1. Partner Robots for Mental Health Care - Japanese Challenge toward Practical Robots - GoroObinata EcoTopia Science Institute, Nagoya University, Nagoya, Aichi 464-8603, JapanE-mail: obinata@mech.nagoya-u.ac.jp

2. JapaneseChallenge in Service Robots Paro ( seal robot ) Wakamaru Muu ifbot Robot for tea service One craftsman still does his work on making these kinds of robots. AIBO

3. Paro “Most successful therapeutic robot” by Dr. Shibata • 350,000 \ = 2,700 UK pounds • 2.7 kg, 9 W • Speaker, Microphones (3), • Sensors Tactile (12), Orientation (3D), Temperature, • Photo • Actuators (7) Eyelid, Head, Limbs • Operating time • 1.5 hoursper one charge • ◆ good selling • (more than 500 Paros) http://paro.jp

4. Alternatives for dogs in animal therapy • Big effects can be seen in Paro. for patients of Alzheimer. for assisting care-givers, • Some trials have been conducted for children of autism or Down syndrome. • ・・・　results in reducing the care costs, reducing the quantity of consumed medicine. • However, it is only the assists to care-giversor more？？？

5. Paro - culture gap - Principle components analysis on questionnaire surveys

6. Summary on Paro • Non-verbal communication of Paro has a big effect with patients of Alzheimer, like animal therapy. • Paro also works good with people in emergency evacuation area. • Paro gives frequently topics of conversation. • Paro plays good with care-givers as the interposition between the patients and care-givers. • The cost merit of Paro in such situations is quite big.

7. Wakamaru-Mitsubishi Heavy Industry with Prof. Ishiguro- Geminoid http://www.irc.atr.jp/Geminoid/

8. Wakamaru • It can be a guide in hospital, or museum, etc. • It can be a cast mate to human actors. • However, it does not seem to be real communication.

9. Summary on Wakamaru • Wakamaru can work for guiding visitors in museums. • The conversation strategy is programmed by a scenario. People can not feel the reality. • Wakamaru can serve if the situation fits.

10. Muu- Research on metamessage or another topics of communication - by Prof. Okada (Toyohashi) It is a good tool for research.

11. ifbot- Our trials for elderly and autistic children - • Undergraduate students • Elderly people who live alone • Autistic children

12. ワーイ 悲しい！ 知らない！ 恥ずかしい！ アアー眠い しょんぼり ifbot 500,000 Yen = 3,900 UK pounds Sale 100 Sensor (impact) Microphone 1 Microphone 2 CCD camera Sensor (distance) Sensor (tactile) Speaker Sensor (distance) This robot can talk with individuals using the voice recognition system. It can express simple emotions with LED on a head, eyes, ears, and mouth or movement of arms. It also can respond toward individuals automatically. Wheel (foot)

13. Young undergraduate students with ifbot- Getting the impression from a robot - • Compare the emotional intensity when a young student use ifbot or computer. • The given task is to search required information. • 64 students ( 49 male, 13 female) Search by Q&A with ifbot Search by using web search engine

14. Questionnaire after the tasks The questionnaire consists of 40 questions, which are categorized by the following ① Blues and Anxiety ② Hostile feeling ③ Boredom ④ Active pleasurable sensation ⑤ Passive pleasurable sensation ⑥ Affinity ⑦ Concentration ⑧ Surprisal The subject selected one from four degrees in each question

15. The subjects feel more something in robot than in web browser ① Blues and Anxiety ① Blues and Anxiety ② Hostile feeling ⑧ Surprisal ⑧ Surprisal ② Hostile feeling ③ Boredom ③ Boredom ⑦ Concentration ⑦ Concentration ④ Activepleasurable sensation ④ Activepleasurable sensation ⑥ Affinity ⑥ Affinity ⑤ Passive pleasurable sensation ⑤ Passive pleasurable sensation ifbot Web browser

16. Monitoring daily living of two weeks with ifbot.The comparison in cases of elderly people who live alone and live with their family. • 6 elderly people spent two weeks with ifbot, and 90 min talk per day in average. 3 persons thought it is a toy. : Elderly people with family 2 persons treated as a conversation partner. 1 person feel it is a kind of child or grandchild.

17. Monitoring daily lives of two weeks with ifbot.Twp elderly people who live alone answered higher rate (90%) than the correct rate (57%) to the question for the recognition of words in ifbot.

18. Autistic children with ifbot • Study 1 explores a relationship between the feature of toys which autistic children usually play with and the development of the children’s social skills through a questionnaire answered by their parents. • Study 2 investigates the effect of verbal communication which the robot conveys on autistic children’s attention and motivation to join training. * The experimenter manipulated the response of ifbot in this study.

19. Study 1 Feature of Toy and Development of Social Skills • Procedure: 19 parents of autistic children (age 6 to 17) participate in this survey. The parents are asked to indicate the degree of social skill which their child is capable and features of a toy which their child prefer to play with in daily life .

20. Study 1 Feature of Toy and Development of Social Skills • Questionnaire: In section 1, the parents respond to 4 items which are based on Social Responsiveness Scale (SRS) , using 4 point scale (1= not true, 2 = sometimes true, 3 = often true, 4 = almost always true); • eye contact (“does he or she make eye contact?”), • gesture (“does he or she use gestures to express his or her feeling?”), • facial expression (“does expression on his or her face match what he or she is saying?”), • language (“Does he or she use language to express his or her feeling?”)

21. Study 1 Feature of Toy and Development of Social Skills • Questionnaire: In section 2, the parents are asked to refer the feature of their children’s favorite toy. Then, they judged whether the toy contains following 4 features or not; face, movement of limb, sound function, and wheel.

22. Results of Study 1

23. Results of Study 1

24. Study2:Autistic Children and Verbal Communication Robot • Procedure: Participants were asked to engage in two interaction trials. In a trial, the participants played a facial expression game with communication robot, which played “instructor” of the game. In another trial, they engaged in the same game with the experimenter, who played same role.  The facial expression game designed to be a tool to determine whether their behavior was identical with verbal information. At the beginning of the trial, we gave time to him or her to adapt new environment. Then, ifbot asked him or her to express the emotion which it inquired by verbal information and ifbot also made the correct facial expression.   ifbot

25. Study2:Autistic Children and Verbal Communication Robot • Measurements: (1) The number of times making eye-contact toward the instructor (2) The number of times responding in verbal communication by instructor (3) Score of facial expression game The number of times making eye-contact and responding in verbal communication are measured when those performance happen during the time we give the children before the facial expression game for adapting to new environment including robot. The score of eye contact and response in verbal communication are evaluated by experimenter using 4 point scale

26. Study2:Autistic Children and Verbal Communication Robot • Measurements: (1) The number of times making eye-contact toward the instructor (2) The number of times responding in verbal communication by instructor (3) Score of facial expression game

27. Results of Study 2 • Case 1. H is a girl, 12 years old, extremely quiet and reserved, who usually avoids eye contact with others. In a trial with the experimenter, at first time the experimenter talked to her, she seemed distracted and didn’t even look at the experimenter. Eventually, she didn’t speak at all until the end of trial. In a trial with robot instructor, in a few moments she met ifbot, she tried to touching ifbot’s face and was looking at upside of ifbot’s head for a while. However, when ifbot started to conversation, she almost didn’t respond and turn her posture to other direction. At the moment ifbot raise its arms, she was back to the facial expression game and answered to Yes/No question at least.

28. Results of Study 2 • Case 2. O is a boy, 11 years old, more active than other participants. His trial started to interact with ifbot first. When O was introduced ifbot, he was very excited and interested in ifbot. He was good at making facial expression and talking with ifbot, but he didn’t respond when ifbot asked a question was exactly same meaning in a different way. After that happened, he was getting distracted. A trial with the experiment was ended very quickly and wasn’t completed. He seemed to know that the trial will be going in the same way just like before he was done with ifbot and kept saying that the experiment is not a robot.

29. Results of Study 2 • Case 3.M is a boy, 8 years old, active and usually talking to himself. At the beginning of trial, Y was being at distance, but when ifbot moved, he was exploring very closely ifbot face features in a few moments and also touched ifbot’s arm several times. He didn’t make eye-contact with ifbot, even talking to it. As time went on, it seemed he became lost his interest to ifbot and started talking to himself again. In a trial with the experiment, Y was sitting very close to experimenter but he didn’t make eye-contact and was talking himself. He continued talking to himself, even the facial expression game started.

30. Results of Study 2 • Trends of all participants: The attention level which included the number of making eye-contact, the number of response in verbal communication and facial expression game score of each child. (Each item response is scored: 1=Very poor, 2 =Poor, 3 =Good, 4 = Very good), Facial expression game score is calculated by summing up the each scores of four tasks. (Each score is coded as 0=no response, 1=response)

31. SummaryAutistic children with ifbot • We found a robot for autism therapy needs to contain a face and movement of limbs based on Study 1. • The results of Study 1 showed sound function doesn’t have the effect on communication skills. • Verbal function of robot is required with more systematic approach for adapting as an element of the robot from Study 2 .

32. SummaryAutistic children with ifbot • We found a robot for autism therapy needs to contain a face and movement of limbs based on Study 1. • The results of Study 1 showed sound function doesn’t have the effect on communication skills. • Verbal function of robot is required with more systematic approach for adapting as an element of the robot from Study 2 .

33. SummaryOur research in mental problems with ifbot • ifbot stimulates more user’s feelings in comparison with computers when the users conduct search tasks. Note that the subjects were familiar with web search but were not with ifbot. • Elderly people who live alone tend to understand the conversation with ifbot is real . The proof appears in their wrong answers to the question of recognition rate of words. • Ifbot has a big potential for curing autistic children. We regard that the face and movement of limb are the keys of imitation and the imitation might lead to improve autistic children’s communication skills.

34. Discussion “Uncanny valley” proposed by Prof. M. Mori in 1976 Human Affinity Uncanny valley a toy doll Degree of similarity to human For an example: moving corpse No one knows the exact axes and the shape of function.

35. Discussion What is the adequate feature of robot for children with autism in robot-assisted therapy or education? Human Effecton Improvement of communication skill Valley ? a good robot for improving communication skill of autistic children a toy doll Imitation level or function of communication

36. Discussion ◆　Communication Robots can be of service for people with appropriate therapists, or care-givers, or teachers if there exists the necessity . We can find the cost benefit in such cases. ◆ We cannot identify the origin of such effects or differences with communication robots. At this stgae, there exist a lot of discussions on the requirements for the robots and the design specifications which meet those requirements. Further research will be required.

37. Thank you. Questions? • These experiments described in this presentation were partly supported by Japanese Ministry of Education, and were conducted with my colleagues Dr. K. Matsumoto, Dr. H. Takehashi, T. Suzuki, N. Takeda and J. Lee. Iwould like to acknowledge them.