1 / 83

Embodied Construction Grammar in language (acquisition and) use

Embodied Construction Grammar in language (acquisition and) use. Jerome Feldman (jfeldman@icsi.berkeley.edu) Computer Science Division, University of California, Berkeley, and International Computer Science Institute. State of the Art. Limited Commercial Speech Applications

scot
Download Presentation

Embodied Construction Grammar in language (acquisition and) use

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Embodied Construction Grammar in language (acquisition and) use Jerome Feldman (jfeldman@icsi.berkeley.edu) Computer Science Division, University of California, Berkeley, and International Computer Science Institute

  2. State of the Art • Limited Commercial Speech Applications transcription, simple response systems • Statistical NLP for Restricted Tasks tagging, parsing, information retrieval • Template-based Understanding programs expensive, brittle, inflexible, unnatural • Essentially no NLU in HCI, QA Systems

  3. “Harry walked to the cafe.” “Harry walked into the cafe.” CAFE CAFE What does language do? A sentence can evoke an imagined scene and resulting inferences: • Goal of action = at cafe • Source = away from cafe • cafe = point-like location • Goal of action = inside cafe • Source = outside cafe • cafe = containing location

  4. Language understanding (Utterance, Situation) Conceptual knowledge Linguistic knowledge Analysis Interpretation

  5. Cafe Language understanding: analysis & simulation “Harry walked to the cafe.” Utterance Lexicon Constructicon Analysis Process General Knowledge Semantic Specification Schema Trajector Goal walk Harry cafe Belief State Simulation

  6. Interpretation: x-schema simulation Constructions can • specify which schemas and entities are involved in an event, and how they are related • profile particular stages of an event • set parameters of an event walker at goal energy goal=home walker=Harry Harryiswalkinghome.

  7. Traditional Levels of Analysis Pragmatics Semantics Syntax Morphology Phonetics

  8. “Harry walked into the cafe.” Pragmatics Semantics Utterance Syntax Morphology Phonetics

  9. Trajector Source Goal Path Construction Grammar A construction is a form-meaning pair whose properties may not be strictly predictable from other constructions. (Construction Grammar, Goldberg 1995) Form Meaning block walk to

  10. Form phonological cues word order intonation inflection Meaning event structure sensorimotor control attention/perspective social goals... Cafe Form-meaning mappings for language Linguistic knowledge consists of form-meaning mappings:

  11. Constructions as maps between relations Complex constructions are mappings between relations in form and relations in meaning. Form Meaning • Mover + Motion • before(Mover, Motion) MotionEventmover(Motion, Mover) “is” + Action+ “ing”before(“is”, Action) suffix(Action, “ing”) ProgressiveActionaspect(Action, ongoing) DirectedMotionEventdirection(Motion, Direction) mover(Motion, Mover) Mover+Motion+Directionbefore(Motion, Direction) before(Mover, Motion)

  12. Embodied Construction Grammar(Bergen and Chang 2002) • Embodied representations • active perceptual and motor schemas • situational and discourse context • Construction Grammar • Linguistic units relate form and meaning/function. • Both constituency and (lexical) dependencies allowed. • Constraint-based (Unification) • based on feature structures (as in HPSG) • Diverse factors can flexibly interact.

  13. Representing image schemas schema name schemaSource-Path-Goal roles source path goal trajector schemaContainer roles interior exterior portal boundary role name Boundary Interior Trajector Portal Source Goal Path Exterior These are abstractions over sensorimotor experiences.

  14. Hypothesis: Linguistic input is converted into a mental simulation based on bodily-grounded structures. Components: Semantic schemas image schemas and executing schemas are abstractions over neurally grounded perceptual and motor representations Linguistic units lexical and phrasal construction representations invoke schemas, in part through metaphor Inferencelinks these structures and provides parameters for a simulation engine Inference and Conceptual Schemas

  15. Early ExampleUnderstanding News Stories France fell into recession. Pulled out by Germany In1991, India set outon a path of liberalization. The Government started to loosen its stranglehold on business and removed obstacles to international trade. Now the Government is stumbling in implementing the liberalization plan.

  16. Task • Interpret simple discourse fragments/blurbs • France fell into recession. Pulled out by Germany • Economy moving at the pace of a Clinton jog. • US Economy on the verge of falling back into recession after moving forward on an anemic recovery. • Indian Government stumbling in implementing Liberalization plan. • Moving forward on all fronts, we are going to be ongoing and relentless as we tighten the net of justice. • The Government is taking bold new steps. We are loosening the stranglehold on business, slashing tariffs and removing obstacles to international trade.

  17. Indian Government stumbling in implementing liberalization plan I/O as Feature Structures

  18. CAFE Language understanding: analysis & simulation constructionWALKED form selff.phon [wakt] meaning : Walk-Action constraints selfm.time before Context.speech-time selfm..aspect  encapsulated “Harry walked into the cafe.” Utterance Analysis Process Constructions General Knowledge Semantic Specification Belief State Simulation

  19. Embodied Construction Grammar providesformal tools for linguistic description and analysis motivated largely by cognitive/functional concerns. • Allows precise specifications of structures/processes involved in acquisition of early constructions • Embodied constructions (structured maps between form and meaning); lexically specific and more general • Usage-based processes of learning new constructions to account for co-occurring utterance-situation pairs • Bridge to detailed psycholinguistic and neural imaging experiments

  20. Formal Cognitive Linguistics • Schemas and frames • Image schemas, force dynamics, executing schemas… • Constructions • Lexical, grammatical, morphological, gestural… • Maps • Metaphor, metonymy, mental space maps… • Mental spaces • Discourse, hypothetical, counterfactual…

  21. CAFE Embodied constructions Notation Form Meaning constructionHARRY form : [hEriy] meaning : Harry Harry constructionCAFE form : [khaefej] meaning : Cafe cafe Constructions have form and meaning poles that are subject to type constraints.

  22. Schema Formalism SCHEMA <name> SUBCASE OF <schema> EVOKES <schema> AS <local name> ROLES < self role name>: <role restriction> < self role name> <-> <role name> CONSTRAINTS <role name> <- <value> <role name> <-> <role name> <setting name> :: <role name> <-> <role name> <setting name> :: <predicate> | <predicate>

  23. A Simple Example SCHEMA hypotenuse SUBCASE OF line-segment EVOKES right-triangle AS rt ROLES Comment inherited from line-segment CONSTRAINTS SELF <-> rt.long-side

  24. Source-Path-Goal SCHEMA: spg ROLES: source: Place path: Directed Curve goal: Place trajector: Entity

  25. Translational Motion SCHEMA translational motion SUBCASE OF motion EVOKES spg AS s ROLES mover <-> s.trajector source <-> s.source goal <-> s.goal CONSTRAINTS before:: mover.location <-> source after:: mover.location <-> goal

  26. Construction Formalism CONSTRUCTION<name> SUBCASE OF <construction> CONSTRUCTIONAL EVOKES <construction> AS <local name> CONSTITUENTS < local name> : <construction> CONSTRAINTS // as in SCHEMAs FORM ELEMENTS CONSTRAINTS // as in SCHEMAs MEANING // as in SCHEMAs

  27. Representing constructions: TO constructionTO form selff.phon [thuw] meaning evokes Trajector-Landmark as tl Source-Path-Goal as spg constraints: tl.trajector«spg.trajector tl.landmark«spg.goal local alias identification constraint The meaning pole may evoke schemas (e.g., image schemas) with a local alias. The meaning pole may include constraints on the schemas (e.g., identification constraints «).

  28. The INTO construction constructionINTO form selff.phon [Inthuw] meaning evokes Trajector-Landmark as tl Source-Path-Goal as spg Container as cont constraints: tl.trajector«spg.trajector tl.landmark«cont cont.interior«spg.goal cont.exterior«spg.source • TO vs. INTO: • INTO adds a Container schema and appropriate bindings.

  29. Grammatical Construction Example CONSTRUCTION Spatial-PP SUBCASE OF Phrase CONSTRUCTIONAL CONSTITUENTS rel: Spatial-Preposition lm: Referring-Exp CONSTRAINTS rel.case <-> lm.case FORM rel < lm MEANING CONSTRAINTS rel.landmark <-> lm

  30. The DIRECTED-MOTIONconstruction constructionDIRECTED-MOTION constructional constituents mover : Thing motion : Motion-Process direction : Source-Path-Goal form moverfbefore motionf motionfbefore directionf meaning evokes Motion-Event as m m.mover « moverm m.motion «motionm m.path «directionm directionm.trajector « moverm motionm.mover « moverm

  31. Semantic specification • The analysis process produces a semantic specification that • includes image-schematic, motor control and conceptual structures • provides parameters for a mental simulation

  32. Language Understanding Process

  33. Constructional analysis

  34. Semantic Specification

  35. CAFE Language understanding: analysis & simulation constructionWALKED form selff.phon [wakt] meaning : Walk-Action constraints selfm.time before Context.speech-time selfm..aspect  encapsulated “Harry walked into the cafe.” Utterance Analysis Process Constructions General Knowledge Semantic Specification Belief State Simulation

  36. The scientist walkedinto the laboratory. The scientistwalkedinto the wall. LAB WALL Bonk!! Simulation-based sense disambiguation Ease of construing nominal as a CONTAINER determines what sense of intois appropriate: • CONTAINER sense • CONTACT sense

  37. The teacher drifted into the house. The smokedrifted into the house. Simulation-based inference Detailed inferences can result from simulation. Image-schematic content of prepositions must fit with properties of other elements of sentence. • Final location of Trajector = inside cafe • Portal = door • Final location of Trajector = inside (possibly throughout) cafe • Portal = door/window

  38. World knowledge informs simulation Physical knowledge of how people and gases interact with houses determines: • Relation between Trajector and Interior The smokedrifted into the house and filled it. ?The teacherdrifted into the house and filled it. • Portal for motion across Boundary The smoke drifted into the house because the window had been left open. ?The teacherdrifted into the house because the window had been left open.

  39. Getting From the Utterance to the SemSpecJohno Bryant • Need a grammar formalism • Embodied Construction Grammar (Bergen & Chang 2002) • Need new models for language analysis • Traditional methods too limited • Traditional methods also don’t get enough leverage out of the semantics.

  40. Embodied Construction Grammar • Semantic Freedom • Designed to be symbiotic with cognitive approaches to meaning • More expressive semantic operators than traditional grammar formalisms • Form Freedom • Free word order, over-lapping constituency • Precise enough to be implemented

  41. Traditional Parsing Methods Fall Short • PSG parsers too strict • Constructions not allowed to leave constituent order unspecified • Traditional way of dealing with incomplete analyses is ad-hoc • Making sense of incomplete analyses is important when an application must deal with “ill-formed” input (For example, modeling language learning) • Traditional unification grammar can’t handle ECG’s deep semantic operators.

  42. Our Analyzer • Replaces the FSMs used in traditional chunking (Abney 96) with much more powerful machines capable of backtracking called constructionrecognizers • Arranges these recognizers into levels just like in Abney’s work • But uses a chart to deal with ambiguity

  43. Our Analyzer (cont’d) • Uses specialized feature structures to deal with ECG’s novel semantic operators • Supports a heuristic evaluation metric for finding the “right” analysis • Puts partial analyses together when no complete analyses are available • The analyzer was designed under the assumption that the grammar won’t cover every meaningful utterance encountered by the system.

  44. Semantic Chunker Semantic Integration System Architecture Grammar/Utterance Learner Chunk Chart Ranked Analyses

  45. The Levels • The analyzer puts the recognizer on the level assigned by the grammar writer. • Assigned level should be greater than or equal to the levels of the construction’s constituents. • The analyzer runs all the recognizers on level 1, then level 2, etc. until no more levels. • Recognizers on the same level can be mutually recursive.

  46. Recognizers • Each Construction is turned into a recognizer • Recognizer = active representation • seeks form elements/constituents when initiated • Unites grammar and process - grammar isn’t just a static piece of knowledge in this model. • Checks both form and semantic constraints • Contains an internal representation of both the semantics and the form • A graph data structure used to represent the form and a feature structure representation for the meaning.

  47. Recognizer Example Mary kicked the ball into the net. This is the initial Constituent Graph for caused-motion. Patient Agent Action Path

  48. Recognizer Example Construct: Caused-Motion Constituent: Agent Constituent: Action Constituent: Patient Constituent: Path The initial constructional tree for the instance of Caused-Motion that we are trying to create.

  49. Recognizer Example

More Related