1 / 150

Towards Robot Theatre

Towards Robot Theatre. Marek Perkowski Department of Electrical and Computer Engineering, Portland State University, Portland, Oregon, 97207-0751. Humanoid Robots and Robot Toys. Talking Robots. Many talking robots exist, but they are still very primitive Work with elderly and disabled

clodia
Download Presentation

Towards Robot Theatre

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards Robot Theatre Marek Perkowski Department of Electrical and Computer Engineering, Portland State University, Portland, Oregon, 97207-0751

  2. Humanoid Robots and Robot Toys

  3. Talking Robots • Many talking robots exist, but they are still very primitive • Work with elderly and disabled • Actors for robot theatre, agents for advertisement, education and entertainment. • Designing inexpensive natural size humanoid caricature and realistic robot heads Dog.com from Japan We concentrate on Machine Learning techniques used to teach robots behaviors, natural language dialogs and facial gestures. Work in progress

  4. Robot with a Personality? • Future robots will interact closely with non-sophisticated users, children and elderly, so the question arises, how they should look like? • If human face for a robot, then what kind of a face? • Handsome or average, realistic or simplified, normal size or enlarged? • The famous example of a robot head • is Kismet from MIT. • Why is Kismet so successful? • We believe that a robot that will interact with humans should have some kind of “personality” and Kismet so far is the only robot with “personality”.

  5. Robot face should be friendly and funny The Muppets of Jim Henson are hard to match examples of puppet artistry and animation perfection. We are interested in robot’s personality as expressed by its: • behavior, • facial gestures, • emotions, • learned speech patterns.

  6. Behavior, Dialog and Learning • Robot activity as a mapping of the sensed environment and internal states to behaviors and new internal states (emotions, energy levels, etc). • Our goal is to uniformly integrate verbal and non-verbal robot behaviors. • Words communicate only about 35 % of the information transmitted from a sender to a receiver in a human-to-human communication. • The remaining information is included in para-language. • Emotions, thoughts, decision and intentions of a speaker can be recognized earlier than they are verbalized. NASA

  7. Morita’s Theory

  8. Our Base Model and Designs

  9. Neck and upper body movement generation

  10. Robot Head Construction, 1999 High school summer camps, hobby roboticists, undergraduates Furby head with new control Jonas We built and animated various kinds of humanoid heads with from 4 to 20 DOF, looking for comical and entertaining values.

  11. Mister Butcher Latex skin from Hollywood 4 degree of freedom neck

  12. Robot Head Construction, 2000 Skeleton Alien We use inexpensive servos from Hitec and Futaba, plastic, playwood and aluminum. The robots are either PC-interfaced, use simple micro-controllers such as Basic Stamp, or are radio controlled from a PC or by the user.

  13. Technical Construction, 2001 Details Marvin the Crazy Robot Adam

  14. 2001 Virginia Woolf heads equipped with microphones, USB cameras, sonars and CDS light sensors

  15. 2002 BUG (Big Ugly Robot) Max Image processing and pattern recognition uses software developed at PSU, CMU and Intel (public domain software available on WWW). Software is in Visual C++, Visual Basic, Lisp and Prolog.

  16. Visual Feedback and Learning based on Constructive Induction 2002 Uland Wong, 17 years old

  17. 2002, Japan Professor Perky Professor Perky with automated speech recognition (ASR) and text-to-speech (TTS) capabilities • We compared several commercial speech systems from Microsoft, Sensory and Fonix. • Based on experiences in highly noisy environments and with a variety of speakers, we selected Fonix for both ASR and TTS for Professor Perky and Maria robots. • We use microphone array from Andrea Electronics. 1 dollar latex skin from China

  18. Maria, 2002/2003 20 DOF

  19. location of head servos Construction details of Maria skull • location of controlling rods • location of remote servos Custom designed skin

  20. Animation of eyes and eyelids

  21. Cynthia, 2004, June

  22. Currently the hands are not moveable. We have a separate hand design project.

  23. Software/Hardware Architecture • Network- 10 processors, ultimately 100 processors. • Robotics Processors. ACS 16 • Speech cards on Intel grant • More cameras • Tracking in all robots. • Robotic languages – Alice and Cyc-like technologies.

  24. Face detection localizes the person and is the first step for feature and face recognition. Acquiring information about the human: face detection and recognition, speech recognition and sensors.

  25. Face features recognition and visualization.

  26. Use of Multiple-Valued (five-valued) variablesSmile, Mouth_Open and Eye_Brow_Raise for facial feature and face recognition.

  27. HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004 Czy znacie dobra sztuke dla teatru robotow? Sonbi, the Confucian Scholar Paekchong, the bad butcher

  28. Editing movements

  29. Yangban the Aristocrat and Pune his concubine The Narrator

  30. The Narrator

  31. We base all our robots on inexpensive radio-controlled servo technology.

  32. We are familiar with latex and polyester technologies for faces Martin Lukac and Jeff Allen wait for your help, whether you want to program, design behaviors, add muscles, improve vision, etc.

  33. New Silicone Skins

  34. A simplified diagram of software explaining the principle of using machine learning based on constructive induction to create new interaction modes of a human and a robot.

  35. Probabilistic and Finite State Machines

  36. Probabilistic State Machines to describe emotions “you are beautiful” / ”Thanks for a compliment” P=1 Happy state “you are blonde!” / ”I am not an idiot” P=0.3 “you are blonde!” / Do you suggest I am an idiot?” Unhappy state P=0.7 Ironic state

  37. Facial Behaviors of Maria Do I look like younger than twenty three? Maria asks: Response: • “no” • “no” • “yes” 0.7 0.3 Maria smiles Maria frowns

  38. Probabilistic Grammars for performances Speak ”Professor Perky”, blinks eyes twice P=0.1 Speak ”Professor Perky” Where? P=0.3 Who? P=0.5 P=0.5 P=0.5 Speak “in some location”, smiles broadly Speak “In the classroom”, shakes head Speak ”Doctor Lee” What? P=0.1 P=0.1 P=0.1 Speak “Was singing and dancing” P=0.1 Speak “Was drinking wine” ….

  39. Human-controlled modes of dialog/interaction Human teaches “Thanks, I have a lesson” “Hello Maria” “Lesson finished” Robot performs Robot asks “Question” “Stop performance” “Questioning finished” “Command finished” “Thanks, I have a question” “Thanks, I have a command” Human asks Human commands

  40. Dialog and Robot’s Knowledge

  41. Robot-Receptionist Initiated Conversation Human Robot What can I do for you? Robot asks This represents operation mode

  42. Robot-Receptionist Initiated Conversation Human Robot What can I do for you? I would like to order a table for two Robot asks

  43. Robot-Receptionist Initiated Conversation Human Robot Smoking or non-smoking? Robot asks

  44. Robot-Receptionist Initiated Conversation Human Robot Smoking or non-smoking? I do not understand Robot asks

  45. Robot-Receptionist Initiated Conversation Human Robot Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. Robot asks

  46. Robot-Receptionist Initiated Conversation Human Robot Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. A table near the terrace, please Robot asks

  47. Human-Initiated Conversation Human Robot Hello Maria initialization Robot asks

  48. Human-Initiated Conversation Human Robot Hello Maria What can I do for you? Robot asks

More Related