1 / 26

Characterizing and Processing Robot-Directed Speech

Characterizing and Processing Robot-Directed Speech. Paulina Varchavskaia • Paul Fitzpatrick • Cynthia Breazeal MIT AI Lab Humanoid Robotics Group. Characterizing and Processing Robot-Directed Speech. Paulina Varchavskaia • Paul Fitzpatrick • Cynthia Breazeal

odessa-kaufman
Download Presentation

Characterizing and Processing Robot-Directed Speech

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Characterizing and Processing Robot-Directed Speech Paulina Varchavskaia • Paul Fitzpatrick • Cynthia Breazeal MIT AI Lab Humanoid Robotics Group

  2. Characterizing and ProcessingRobot-Directed Speech Paulina Varchavskaia • Paul Fitzpatrick • Cynthia Breazeal MIT AI Lab Humanoid Robotics Group

  3. Characterizing and Processing Robot-Directed Speech Paulina Varchavskaia • Paul Fitzpatrick • Cynthia Breazeal MIT AI Lab Humanoid Robotics Group

  4. Characterizing and ProcessingRobot-Directed Speech Paulina Varchavskaia • Paul Fitzpatrick • Cynthia Breazeal MIT AI Lab Humanoid Robotics Group

  5. Speech Recognition • What speech does the robot evoke? • How can we process that speech? • Support a growing vocabulary • Without hurting recognition accuracy

  6. Extending Vocabulary /lft/ /ryt/ /yn/ /ylo/ /ylo/ /ylo/ /ylo/ /ylo/ /ylo/ /ylo/ /ylo/ /ylo/ /ylo/ /lft/ /lft/ /lft/ /lft/ /lft/ /lft/ /lft/ /lft/ /lft/ /lft/ /ryt/ /lft/ /ryt/ Extending Vocabulary /ryt/ /ryt/ /ryt/ /ryt/ /ryt/ /ryt/ /grin/ /grin/ /grin/ /grin/ /grin/ /grin/ /grin/ /grin/ /ylo/ /ryt/ /lft/ /ryt/ /ylo/ /grin/ Baseline System

  7. Baseline System • “Magic word” to introduce vocabulary • Separation of introduction from use • Robot confused by unknown/filler words

  8. 13 children 5-10 years old 20 minute sessions Some prompting (Turkle et al) Study: Characterizing Speech

  9. 10 9 8 7 6 number of utterances per minute (approx) 5 4 3 2 1 0 0 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 children (cumulative %) Number of Utterances

  10. 100 90 80 70 60 single word utterances (%) 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 children (cumulative %) Single Word Utterances

  11. 100 90 80 70 60 utterances with enunciation (%) 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 children (cumulative %) Utterances with Enunciation

  12. Cooperative Speech Exists • Some clean word-learning scenarios • “Tiffany” • Can we use them as leverage? • “My name is Tiffany” • What recognition rates are possible?

  13. Study: Processing Speech • “LCSINFO” domain (Glass, Weinstein) • Provide some initial vocabulary • Build language model without transcripts

  14. Language Model word 1 word 2 word n

  15. OOV Out Of Vocabulary Model word 1 word 2 word n

  16. phone 1 phone 2 phone n Out Of Vocabulary Model OOV (Bazzi)

  17. Run recognizer Extract OOV fragments Identify rarely-used additions Identify competition Update Language Model Add to lexicon Remove from lexicon Update lexicon, baseforms Hypothesized transcript N-Best hypotheses

  18. Keyword Error Rate (%) coverage (%)

  19. Qualitative Results Given: 1600 Utterances “email, phone, room, office, address” Finds: 1. n ah m b er 6. p l iy z 2. w eh r ih z 7. ae ng k y uw 3. w ah t ih z 8. n ow 4. t eh l m iy 9. hh aw ax b aw 5. k ix n y uw 10. g r uw p

  20. Conclusions • What about acoustic models? • Idiosyncratic vocabulary is important • Pick a good name for your robot!

  21. Acknowledgements • MIT Initiative on Technology and Self • MIT Spoken Language Systems group • DARPA • NTT

More Related