1 / 22

Visual Semantics and Ontology of Eventive Verbs

Visual Semantics and Ontology of Eventive Verbs. Minhua Eunice Ma and Paul Mc Kevitt School of Computing and Intelligent Systems Faculty of Engineering University of Ulster, Magee Derry/Londonderry, N. Ireland. Outline. Background: CONFUCIUS Previous verb taxonomies

adonis
Download Presentation

Visual Semantics and Ontology of Eventive Verbs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visual Semantics and Ontology ofEventive Verbs Minhua Eunice Ma and Paul Mc Kevitt School of Computing and Intelligent Systems Faculty of Engineering University of Ulster, Magee Derry/Londonderry, N. Ireland

  2. Outline • Background: CONFUCIUS • Previous verb taxonomies • Visual semantics & verb classes • CONFUCIUS’ ontology of verbs • Current status of implementation • Relation to other work • Conclusion & future work IJCNLP-04 Sanya, China, 22 Mar 2004

  3. Architecture of CONFUCIUS Natural language sentences Surface transformer Media allocator Prefabricated objects (knowledge base) LCS lexicon WordNet Natural Language Processing Text To Speech Sound effects Language knowledge 3D authoring tools, existing 3D models & character models semantic representations mapping visual knowledge Animation engine Visual knowledge (3D graphic library) Synchronising & fusion 3D world with audio in VRML IJCNLP-04 Sanya, China, 22 Mar 2004

  4. Pre-processing Coreference resolution Part-of-speech tagger Syntactic parser Morphological parser Temporal reasoning Semantic inference NLP in CONFUCIUS Connexor FDG parser FEATURES Disambiguation WordNet LCS database Post-lexical temporal relations Lexical temporal relations IJCNLP-04 Sanya, China, 22 Mar 2004

  5. Previous verb taxonomies • Grammatical categorisation & valency • Thematic roles (Fillmore, 1968; Jackendoff, 1990; Halliday, 1985;Dowty, 1991) • Aspectual classes (Vendler, 1967; Stede, 1996) • Semantic verb classes in WordNet (Fellbaum, 1998) • Levin’s (1993) verb classes • Dimension of causation (Asher & Lascarides, 1995) IJCNLP-04 Sanya, China, 22 Mar 2004

  6. Grammatical categorisation & valency • Subcategorisation description of verb categories in LDOCE (Longman Dictionary of Contemporary English) • D – ditransitive • I – intransitive • L – linking verb with complement • T1 – transitive verb with NP object • T3 – transitive verb with infinitival clause as object IJCNLP-04 Sanya, China, 22 Mar 2004

  7. Grammatical categorisation & valency • Subcategorisation description of verb categories in LDOCE (Longman Dictionary of Contemporary English) • Syntactic valency • Obligatory valency fillers (complements) e.g. subject, object • Optional valency fillers (adjuncts) e.g. temporal, locational adjuncts • Semantic valency (Leech, 1981) IJCNLP-04 Sanya, China, 22 Mar 2004

  8. Thematic roles • Other names: theta-role, case role, deep grammatical function, transitivity role, valency role, case frame • Extend syntactic analysis into semantic domain to capture roles of participants surface case (nominative, accusative) surface function (subject, object) • Classifying verbs based on thematic roles (Dixon, 1991) Thematic roles (e.g. agent, patient/theme, instrument, source, goal, place) IJCNLP-04 Sanya, China, 22 Mar 2004

  9. Aspectual (temporal) classification • Vendler’s (1967) verb classes • activities: run, swim, sleep, cry • statives: love, hate, know • achievements: arrive, win, find, die • accomplishments: build (a house), write (a book) • Stede’s (1996) MOOSE ontology • Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events IJCNLP-04 Sanya, China, 22 Mar 2004

  10. situation state activity event protracted activity moment activity culmination transition protracted culmination moment culmination Aspectual (temporal) classification • Vendler’s (1967) verb classes • Stede’s (1996) ontology of MOOSE • Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events IJCNLP-04 Sanya, China, 22 Mar 2004

  11. situation state activity event protracted activity moment activity culmination transition protracted culmination moment culmination Aspectual (temporal) classification • Vendler’s (1967) verb classes • Stede’s (1996) ontology of MOOSE • Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events IJCNLP-04 Sanya, China, 22 Mar 2004

  12. Semantic verb classes in WordNet • Taxonomic approach based on pure lexical semantics • Reveal semantic organisation of lexicon in terms of lexical & semantic relations • Top nodes of WordNet’s verb file IJCNLP-04 Sanya, China, 22 Mar 2004

  13. Levin’s (1993) verb classes • Theoretic ground -- semantic/syntactic correlations: verbs with similar meaning (identical LCSs in terms of specific meaning components) show same syntactic behaviors Verbs of inherently directed motion: arrive, come, enter Leave verbs: leave, abandon, desert Manner of motion verbs: roll, run, sneak, waddle Verbs of motion using a vehicle: bike, drive, fly Chase verbs: chase, follow, track Accompany verbs: accompany, escort, guide Waltz verbs: clog, polka, waltz Verbs of motion IJCNLP-04 Sanya, China, 22 Mar 2004

  14. Dimension of causation • Asher and Lascarides’ (1995) dimension of causation-change • causation and change are specified along four dimensions: locative, formal, matter, intentional cause locative formal matter intentional change loc-cause sub of put fml-cause sub of build mtr-cause sub of paint intent-cause sub of amuse loc-change obj of put fml-change obj of build intent-change obj of amuse mtr-change obj of paint IJCNLP-04 Sanya, China, 22 Mar 2004

  15. Visual semantics & verb classes Visual factors concerning verb categorisation • Visual valency • Somatotopic factors in visualisation • Level-of-detail of visual information Verbs belonging to same class in the classification • Visual “synonyms” • Substitutable in same set of animation keyframes Visualisation of action verbs is effective evaluation of the classification IJCNLP-04 Sanya, China, 22 Mar 2004

  16. Visual valency • Capacity of verb to take specific number and type of visual arguments in language visualisation (3D animation) • valency filler -- visual role • 2 types of visual roles requiring different processes in visualisation • human (biped articulated animate entity) • object (inanimate entity) • Visual valency overlaps with syntactic & semantic valency • Visual modality requires more obligatory roles than surface grammar or semantics IJCNLP-04 Sanya, China, 22 Mar 2004

  17. facial expression – sing, laugh Leg – run, kick body posture Arm – wave, put Somatotopic effectors of action verbs • Theoretical ground: execution/perception/visualisation of action verbs produced by same somatotopic effector activate same parts of cortex • Distinguish facial expression (e.g. lip movement) & body posture (arm/leg/torso) in our ontological system • Further divisions like distinction between upper/lower arm, hands, & even fingers are possible torso – bow IJCNLP-04 Sanya, China, 22 Mar 2004

  18. EVENT … go cause event level verbs … walk climb run jump manner level verbs limp stride trot swagger troponym level verbs jog romp skip bounce hop Level-Of-Detail (LOD)basic-level verbs & troponyms IJCNLP-04 Sanya, China, 22 Mar 2004

  19. CONFUCIUS’ verb taxonomy 2.2.1. Action verbs 2.2.1.1. One visual valency (the role is a human, (partial) movement) 2.2.1.1.1. Biped kinematics: arm actions (wave, scratch), leg actions (walk, jump, kick), torso actions (bow), combined actions (climb) 2.2.1.1.2. Facial expressions & lip movement, e.g. laugh, fear, say, sing, order 2.2.1.2. Two visual valency (at least one role is human) 2.2.1.2.1. One human and one object (vt. or vi.+instrument) e.g. throw, push, kick, open, eat, drink, bake, trolley 2.2.1.2.2. Two humans, e.g. fight, chase, guide 2.2.1.3. Visual valency ≥ 3 (at least one role is human) 2.2.1.3.1. Two humans and one object (inc. ditransitive verbs), e.g. give, show 2.2.1.3.2. One human and 2+ objects (vt. + object + implicit instr./goal/theme) e.g. cut, write, butter, pocket, dig, cook 2.2.1.4. Verbs without distinct visualisation when out of context: verbs of trying, helping, letting, creating/destroying 2.2.1.5. High level behaviours (routine events), political and social activities e.g. interview, eat out (go to restaurant), go shopping IJCNLP-04 Sanya, China, 22 Mar 2004

  20. Text-to-Animation of single sentences • Collision detection example (contact verbs: hit, collide, scratch, touch), no human role involved • “The car collided with a wall.” • using ParallelGraphics’ VRML extension--object-to-object collision • non-speech sound effects • H-Anim examples: action verbs • 3 visual valency verbs • “John gave Nancy a loaf of bread.” • “John put a cup of coffee on the table.” • H-Anim Site node • locative tags of object (on_table tag for table object) • 2 visual valency verbs • “John pushed the door.” • “John ate the bread.” • “Nancy sat on the chair.” • For more demos, please visit http://www.infm.ulst.ac.uk/~eunice/3D_anim.html IJCNLP-04 Sanya, China, 22 Mar 2004

  21. Relation to other work • Categorise verbs from visual semantics perspective • Language visualisation in CONFUCIUS provides independent criteria for identifying classes of verbs sharing certain aspects of meaning, i.e. semantic/visual correlations • Relation to Levin’s verb classes: “Carol cut the whole wheat bread.” “Whole wheat bread cuts easily.” 2.2.1.3.2, visual valency=3 2.1.2, visual valency=2 Verbs of cutting 1 to N “Nancy brought the book to John.” “Nancy gave the book to John.” Verbs of sending & carrying Verbs of change of possession 2.2.1.3.1 visual valency=3 N to 1 IJCNLP-04 Sanya, China, 22 Mar 2004

  22. Conclusion & future work • Categorise verbs from visual semantic perspective • Provides independent criteria for identifying classes of verbs based on semantic/visual correlations • Visual semantic analysis of eventive verbs revealed striking influences in taxonomic verb tree • Various criteria ranging from visual valency, somatotopic effectors, to LOD are proposed • Evaluation issues using automatic animation generation & psychological experiments • Discourse level interpretation IJCNLP-04 Sanya, China, 22 Mar 2004

More Related