74.406 Natural Language Processing

1. 74.406 Natural Language Processing • Semantics I • General Introduction • Types of Semantics • From Syntax to Semantics • Semantics II • Desiderata for Representation • Logic-based Semantics

2. Semantics I

3. Semantics Distinguish between • surface structure (syntactic structure) and • deep structure (semantic structure) of sentences. Different forms of Semantic Representation • logic formalisms • ontology / semantic representation languages • Case Frame Structures (Filmore) • Conceptual Dependy Theory (Schank) • DL and similar KR languages • Ontologies

4. Semantic Representations Semantic Representation based on some form of (formal) Representation Language: • Semantics Networks • Conceptual Dependency Graphs • Case Frames • Ontologies • DL and similar KR languages

5. Constructing a Semantic Representation General: • Start with surface structure derived from parser. • Map surface structure to semantic structure • Use phrases as sub-structures. • Find concepts and representations for central phrases (e.g. VP, NP, then PP) • Assign phrases to appropriate roles around central concepts (e.g. bind PP into VP representation).

6. Ontology (Interlingua) approach • Ontology: a language-independent classification of objects, events, relations. • A Semantic Lexicon, which connects lexical items to nodes (concepts) in the ontology. • An analyzer that constructs Interlingua representations and selects appropriate one. (based on Steve Helmreich's 419 Class, Nov 2003)

7. Semantic Lexicon • Provides a syntactic context for the appearance of the lexical item. • Provides a mapping for the lexical itemto a node in the ontology (or more complex associations). • Provides connections from the syntactic context to semantic roles and constraints on these roles.

8. Deriving Basic Semantic Dependency Deriving Basic Semantic Dependency (a toy example) Input: John makes tools Syntactic Analysis: cat verb tense present subject   root john cat noun-proper object   root     tool cat noun number plural

9. Lexicon Entries for Johnand tool John-n1 syn-struc root john cat noun-proper sem-struchuman name john gender male tool-n1 syn-struc root tool cat n sem-struc tool

10. Meaning Representation - Example make Relevant Extract from the Specification of the Ontological Concept Used to Describe the Appropriate Meaning of make: manufacturing-activity... agent human theme artifact …

11. Relevant parts of the (appropriate senses of the) lexicon entries for John and tool John-n1 syn-struc root john cat noun-proper sem-struchuman name john gender male tool-n1 syn-struc root tool cat n sem-struc tool

12. Semantic Dependency Component The basic semantic dependency component of the TMR forJohn makes tools manufacturing-activity-7 agent uman-3 theme set-1 element tool cardinality > 1 …

13. try-v3 syn-struc root try cat v subj root $var1 cat n xcomp root $var2 cat v form OR infinitive gerund sem-struc set-1 element-type refsem-1 cardinality >=1 refsem-1 sem event agent ^$var1 effect refsem-2 modality modality-type epiteuctic modality-scope refsem-2 modality-value < 1 refsem-2 value ^$var2 sem event

14. Constructing an IL representation For each syntactic analysis: • Access all semantic mappings and contexts for each lexical item. • Create all possible semantic representations. • Test them for coherency of structure and content.

15. “Why is Iraq developing weapons of mass destruction?”

16. Word sense disambiguation • Constraint checking – making sure the constraints imposed on context are met • Graph traversal • is-a links are inexpensive • otherlinks are more expensive • The “cheapest” structure is the most coherent

17. Semantics II

18. Desiderata for a Semantic Representation • Verifiability – semantic representation must be compatible with knowledge (base) of the system. • Canonical Form - assign same representation to different surface expressions which have essentially the same meaning • Ambiguity and Vagueness – representation should (in relation to knowledge base or information system access etc.) be unambiguous and precise

19. Semantics - Connecting Words and Worlds Semantic Representation NL Input Knowledge Representation World State (KB: T-Box, A-Box) NL Output

20. Representation of Meaning Representation of meaning for natural language sentences: • Semantic Representation Language (in most cases) = some kind of formal language + semantic primitives • For example: First Order Predicate Logicwithspecificset of predicatesandfunctions

21. Semantic Representations Semantic Representation based on some form of (formal) Representation Language. • Semantics Networks • Conceptual Dependency Graphs • Case Frames • Ontologies • DL and similar KR languages • First-Order Predicate Logic

22. Example - NL Database Access Imagine a database access using natural language, i.e. questions to the DB posed in natural language. Example: DB of courses in the CS department Pose questions like: • Who is teaching Advanced AI in Fall 2004? • Is John Anderson teaching this term? • What is John Anderson teaching this term? • Who is teaching AI at the University of Winnipeg? • Who is teaching an AI related course this term?

23. Example Story: My car was stolen two weeks ago. They found it last week. • direct representation of meaning • knowledge • inference

24. Example Primitives in logic language FOPL: my car as individual constant my_car, car_1 can make statement owns(car_1, I) about ownership of car owns(car_1, Speaker) 2-place predicate owns with one place for the object / car and one place for the owner; filled with variable or constant owns(car_1, Speaker) Someone owns car_1.x: owns(car_1, x) I own all cars.x: car(x)  owns(x, Speaker)

25. Example Primitives in logic language FOPL: stolen as predicate applied to car stolen(car_1) as event, specified with variable for event and constant for specific event stolen-evente,x: event(e)  stolen(e,x)  x= car_1 ore,x: event(e)  stolen(e, car_1) can make additional specifications, e.g.tense; time; location e,x: event(e)  stolen(e, car_1)  past(e)  time(e)=UT-2weeks / time(e,UT-2weeks)  loc(e)=street#1 event time before utterance time utterance time - 2 weeks refers to identified street

26. Example Primitives in logic language FOPL: They found it last week. found(car_1,t)  time(t)  t=(UT-1week)

27. NL and Logic Levels of Representation and Transformation • direct representation of meaning • translation into logic expression • knowledge • stored information about relations etc., e.g. as rules • ontology; terminology; proper axioms • inference • gain additional information, conclusions • combine semantic representation and knowledge

28. Example concrete world description car (my_car) stolen (my_car, t1), owns (speaker, my_car) found (police, my-car, t2) t1<t2 stolen (x, t1) owns (y, x)and found (police, x, t2)implies has (y, x, t3)for some timepointst1, t2, t3witht1<t2<t3 general world knowledge What can you inferif you instantiate x with my_car?

29. Reichenbach's Approach to English Tenses U Time of Utterance R Reference Time E Event Time Fig. 14.4. from Jurafsky and Martin, p. 530

30. Example car (my_car) stolen (my_car, t1), owns (speaker, my_car) found (police, my-car, t2) t1<t2 stolen (x, t1) owns (y, x)and found (police, x, t2)implies has (y, x, t3)for some timepointst1, t2, t3witht1<t2<t3 stolen(my_car, t1)  owns (speaker, my_car)  found (police, my-car, t2)  has (speaker, my_car, t3) Do: pattern matching with variable binding / unification; inference

31. Example stolen (x, t1) owns (y, x) implication? Express that, if something is stolen, the owner does not have it anymore!

32. Predicate-Argument Structure Verb-centered approach Thematic roles, case roles Describe semantic structure based on verb and associated roles filled by other parts of the sentence (phrases). Representation using e.g. logic: • Transform structured input sentence (syntax!) into expression in predicate logic. • Usually based on central predicate, the verb, or equivalent, like ‘be’+ adjective etc. • Other parts of the sentence directly related to the verb go into the central predicate.

33. Verb Subcategorization Consider possible subcatframes of verbs. Example: 3 different kinds of want: • NP want NP I want money. want1 (Speaker, money) • NP want Inf-VP He wants to go home. want2 (he, to_go_home) • NP want NP Inf-VP I want him to go away. want3 (I, him, to_go_away)

34. Example - Restaurant 'Maharani' Example: Restaurant 'Maharani' • Maharani serves vegetarian food. • Maharani is a vegetarian restaurant. • Maharani is close to ICSI. Write down logical formulas representing the three different sentences.

35. Logic Formalisms Lambda Calculus

36. Semantics - Lambda Calculus 1 Logic representations often involve Lambda-Calculus: • represent central phrases (e.g. verb) as -expressions • -expression is like a function which can be applied to terms • insert semantic representation of complement or modifier phrases etc. in place of variables x, y: loves (x, y) FOPL sentence xy loves (x, y)-expression, function xy loves (x, y) (John)  y loves (John, y)

37. Semantics - Lambda Calculus 2 Transform sentence into lambda-expression: “AI Caramba is close to ICSI.” specific: close-to (AI Caramba, ICSI) general: x,y: close-to (x, y)  x=AI Caramba  y=ICSI Lambda Conversion: form -expression: xy: close-to (x, y) (AI Caramba) Lambda Reduction: apply -expression y: close-to (AI Caramba, y) close-to (AI Caramba, ICSI)

38. Semantics - Lambda Calculus 3 • Lambda Expressions as basis for semantic representations • attached to words and syntactic categories in grammar rules • passed between nodes (according to grammar rules) during parsing example: Verb  serve {x y e IS-A(e, Serving)  Server(e, y)  Served(e, x)} represents the semantics of the verb 'serve' Reification denotes the use of predicates as constants. Allows the use of "predicates over predicates", e.g. statements like IS-A (serving, event) or IS-A (restaurant, location). e: serving - event, action, verb reification

39. Semantics - Lambda Calculus 4 Lambda Expressions are constructed from central expression, inserting semantic representations for subject and complement phrases: Verb  serve {x y e IS-A(e, Serving)  Server(e,y)  Served(e, x)} Fill in appropriate expressions fory, x, for example'Ay Caramba'for yand'meat'for x, derived from Noun in NP as subject NP of the sentence, and y as complement NP to the verb. e: serving - S, event y: restaurant - NP, subj. y: person - NP, dir. obj.

40. Semantics - Lambda Calculus 5 Complete semantic representation is produced by combining semantic feature structures of phrases in the sentence, according to extended grammar rules. Verb serves {x y e IS-A(e,Serving) Server(e, y) Served(e, x)} Apply lambda-expression representing the verb semantics to semantic representations of NPs: {x y e IS-A (e,Serving) Server (e, y) Served(e, x)} Successively apply the lambda-expression (for serves in the example above), filling thex-position with the semantics of the object-NP, and the y-position with the representation of the subject-NP.

41. Semantics - Lambda Calculus 6 Extend the grammar with semantic attachments, e.g. NP  ProperNoun {ProperNoun.sem} The "base" semantic attachment is determined through access to a lexicon or an ontology. It corresponds to the concept associated with the lexical word, or in the simplest form just the lexical word. Example:Ay Carambaas individual constant orMeatas (reified) concept.

42. Semantics - Lambda Calculus 7 Constructive Semantics During parsing, these semantic attachments are combined, according to the grammar rules, to form more complete representations, finally covering the whole sentence. Combine and pass upwards semantic attachments, e.g. S  NPVP{VP.sem {NP.sem}} VP  VerbNP{Verb.sem {NP.sem}}

43. Semantic Representation from BeRP Parse tree with semantic attachments for the sentence "AyCaramba serves meat." Figure 15.3. from Jurafsky and Martin, p. 554.

44. Semantics - Lambda Calculus 8 Modifiers can be added into semantic description as part of the grammar rules, by intersection of concepts: Nominal  Adj Nominal {x. Nominal.sem(x) IS-A(x, Adj.sem)} Example: a "cheap restaurant" x. IS-A (x, restaurant) IS-A(x, cheap) Problem if intersection of concepts is misleading, e.g. "former friend". Use "modification" rule instead : Nominal  Adj Nominal {x. Nominal.sem(x) AdjMod(x, Adj.sem)} Use rule for "cheap restaurant": x. IS-A (x, restaurant) AdjMod(x, cheap) where "cheap" modifies the "restaurant" in a specific way.

45. Semantics - Problems Problems with Modal Verbs: • apply to predicate structure (other verb) • referential opaqueness • not standard implications Example: I think Joe's flight leaves at 7pm. (think (Speaker, leaves (Joe's flight, 7 pm))) Add: Joe's flight is BA727. BA 727 is delayed. Add: I think I go home. cannot apply predicate to formula in FOPL What does Speaker think now? Should I stay or should I go?

46. Parsing with Semantic Features Modified Early Algorithm. Figure 15.5. from Jurafsky and Martin, p. 570.

47. References Jurafsky, D. & J. H. Martin, Speech and Language Processing, Prentice-Hall, 2000. (Chapters 9 and 10) Helmreich, S., From Syntax to Semantics, Presentation in the 74.419 Course, November 2003.