1 / 7

Semantic Parsing for Robot Commands

Semantic Parsing for Robot Commands. Justin Driemeyer Jeremy Hoffman. Purpose. STAIR (STanford AI Robot) intelligent office assistant Spoken dialogue, e.g. commands to move “Robot, could you move forward two feet?” “Please go up to my office, room two-oh-four, and wait there.”

earl
Download Presentation

Semantic Parsing for Robot Commands

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Semantic Parsing for Robot Commands Justin Driemeyer Jeremy Hoffman

  2. Purpose • STAIR (STanford AI Robot) • intelligent office assistant • Spoken dialogue, e.g. commands to move • “Robot, could you move forward two feet?” • “Please go up to my office, room two-oh-four, and wait there.” • “Oh, uh, wait, hey robot, stop.” • Goal: robust language understanding • Extract meaning of utterance from transcription • Overcome unexpected words or phrasing, stuttering, errors of the speech recognition component

  3. Dialogue System Audio Input Utterance/ Command Semantics SEMANTIC PARSER DIALOG MANAGER SPEECH RECOGNIZER Text Robot action (e.g., speak, move)

  4. Semantic PCFG Parser • Idea: PCFG parsing with semantic grammar • Tag sentences with semantic labels instead of syntactic labels, e.g., “MoveWord,” “Destination,” “ExtraWord” • Train our Ass’n 3 PCFG parser from a treebank of hand-labeled utterance transcriptions • From a parsed sentence, can read off the important content directly • Problem: no robot-command corpus (yet) • Instead, automatically generate a fake treebank of tagged sentences • We hand-wrote a PCFG for commands to move or control movement • Also serves as “probability prior” once we get some real data

  5. Frames and Slots In Our Grammar • We focused on the three commands encoding movement.

  6. Command Extraction Parse tree: (ROOT (S (MacroMove (MacroMoveWords (MacroMoveWord go) (MacroMoveWord to)) (Destination (NamedPlace (WordThe the) (NamedPlaceWord lab) (NamedPlaceWord room)))))) Desired Command: (MacroMove (Floor 0) (Room 100))

  7. Command Extraction • Two parse trees could be different, but encode the same command: Gold Standard: (ROOT (S (MacroMove (RobotAddress (ExtraWords (ExtraWord hey)) (RobotName (RobotNameWord stair)) (PoliteWords (PoliteWord could) (PoliteWord you))) (MacroMoveWords (MacroMoveWord follow) (MacroMoveWord me) (MacroMoveWord to)) (Destination (NamedPlace (NamedPlaceWord my) (NamedPlaceWord office)))))) Extracted: (ROOT (S (MacroMove (RobotAddress (Politeness (PoliteWords (PoliteWord hey))) (RobotName (RobotNameWord stair)) (Politeness (PoliteWords (PoliteWord could) (PoliteWord you) (PoliteWord follow) (PoliteWord me)))) (MacroMoveWords (MacroMoveWord to)) (Destination (NamedPlace (NamedPlaceWord my) (NamedPlaceWord office)))))) • Both encode: Command Tree: (MacroMove (Floor 2) (Room 243))

More Related