Natural Language Processing

1. Natural Language Processing • what it does • what is involved • why is it difficult • brief history

2. sentence  structured repns of meaning "how old is my help3.doc file?" Lisp: (query (file-detail 'date "C:/help3.doc")) "the large cat chased the rat" Logic: (1s1large(s1)  feline(s1))  (1s2rodent(s2))  chased(s1, s2 ) "the young boy ate a bad apple" CD Graph ...see next page...

3. CD graph "the young boy ate a bad apple"

4. what is involved symbolic computationie: symbols manipulated by symbol processors search & inference knowledge representation techniques

5. why is it difficult prejudice, politics, etc ambiguity... • syntactic • semantic • pragmatic

6. example sentences • the old man the boats • my car drinks petrol • I saw the Eiffel Tower flying to Paris • he opened the door with the keyhe opened the door with the squeaking hinge • the boy kicked the ball under the treethe boy kicked the wall under the tree • put the bottles in the box on the shelf by the door

7. (brief) history of language processing 1950s Russian  English translation 1956 Chomsky 1960s Pattern matching 1970s Parsing & some KnRep 1980s Kn & inference 1990s big dreams small results 2000+ quietly promising

8. matching:Sir

9. matching:Student

10. matching:Elisa

11. input sentence a modernapproach morphological processing lexicon syntax analysis (parsing) grammar semantic rules semantic analysis pragmatic analysis contextual information target representation

12. step 1- morphological processing objective: strip words into roots & modifiers issues • inflection (cat pl cat-s) • derivation (happy adj happiness noun) • compounding (toothpaste)

13. morphological processing - notes • all(?) spoken lngs exhibit morphology • easier to handle in written lngs if not iconic • some morphology describes infm beyond syntax eg: proximity (Tamil, Setswana, etc) case speaker / listener peer relationship

14. morphology examples

15. step 2- syntax analysis objectives: 1 check for correctness 2 produce phrase structure uses • parser a rule-based search engine • grammar context-free production rules • lexicon dictionary of words & their categories

16. syntax rules parts of speech rules of combination consider • the cat chases the mouse • all large black dogs chase cats

17. example 1 - using Lkit (build-lexicon '((a determiner) (cat noun) (dog noun) (the determiner) (chased verb))) (build-grammar '((s1 (sentence -> noun-phrase verb-phrase)) (np (noun-phrase -> determiner noun)) (vp (verb-phrase -> verb noun-phrase)) ))

18. example 1 - output (parse 'sentence '(the dog chased a cat)) complete-edge 0 5 s1 sentence (the dog ...) nil s1 sentence -> (noun-phrase verb-phrase) Syntax (sentence (noun-phrase (determiner the) (noun dog)) (verb-phrase (verb chased) (noun-phrase (determiner a) (noun cat)))) Semantics (sentence)

19. so what ?we want meaning

20. Remember: "the young boy ate a bad apple" how can semantics be encoded as symbols? the boy / an apple? young/old, happy/sad, good/bad? how can semantics be generated? what can be inferred from semantics?

21. Reminder: "the young boy ate a bad apple"

22. symbolic representation of semantics (actor (root boy) (id boy#732) (tags animate human male) (qual (age (val 5) (approx 3))) (quant specific)) (action (primitve INGEST)) (object (root apple) (id nil) (tags physob veg fruit food) (qual (phy-state -4)) (quant non-specific))

23. semantics in lexicon a simple example (build-lexicon '((a det any ) (cat noun feline ) (chased verb hunts ) (dog noun canine ) (the det specific) ))

24. semantics in grammar rules (s1 (sentence -> noun-phrase verb-phrase) (actor . noun-phrase) (action . verb-phrase.action) (object . verb-phrase.object) ) (np (noun-phrase -> det noun) (det . noun) ) (vp (verb-phrase -> verb noun-phrase) (action . verb) (object . noun-phrase) )

25. semantics - results (parse 'sentence '(the dog chased a cat)) complete-edge 0 5 s1 sentence (the dog...) nil s1 sentence -> (noun-phrase verb-phrase) Syntax (sentence (noun-phrase (det the) (noun dog)) (verb-phrase (verb chased) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor (specific canine)) (action hunts) (object (any feline)))

26. semantics in lexicon - checks 1 (a det (sems . any)) (all det (sems . every)) (cat noun (sems . feline) (num . sing)) (cats noun (sems . feline) (num . plur)) (chase verb (sems . hunts) (num . plur)) (chases verb (sems . hunts) (num . sing)) (dog noun (sems . canine) (num . sing)) (dogs noun (sems . canine) (num . plur)) (the det (sems . specific))

27. semantics in grammar - checks 1 (s1 (sentence -> noun-phrase verb-phrase) (actor . noun-phrase.sems) (action . verb-phrase.action) (object . verb-phrase.object) ; check number of noun-phrase & verb-phrase (if (noun-phrase.number = verb-phrase.number) numeric-agreement-ok numeric-agreement-bad ) )

28. semantics - results (parse 'sentence '(the dog chases a cat)) complete-edge 0 5 s1 sentence (the dog...) nil s1 sentence -> (noun-phrase verb-phrase) Syntax (sentence (noun-phrase (det the) (noun dog)) (verb-phrase (verb chases) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor specific canine) (action . hunts) (object any feline) numeric-agreement-ok)

29. semantics - results (parse 'sentence '(the dogs chases a cat)) complete-edge 0 5 s1 sentence (the dog...) nil s1 sentence -> (noun-phrase verb-phrase) Syntax (sentence (noun-phrase (det the) (noun dog)) (verb-phrase (verb chases) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor specific canine) (action . hunts) (object any feline) numeric-agreement-bad)

30. semantics in grammar - checks 2 (s1 (sentence -> noun-phrase verb-phrase) (fail if noun-phrase.number /= verb-phrase.number) (actor . noun-phrase.sems) (action . verb-phrase.action) (object . verb-phrase.object) )

31. semantics - results (parse 'sentence '(the dog chases a cat)) Semantics (sentence (actor specific canine) (action . hunts) (object any feline)) (parse 'sentence '(the dogs chases a cat)) .... failed ....

32. semantics in grammar - checks 3 (s1 (sentence -> noun-phrase verb-phrase) (glitch numeric-agreement if not noun-phrase.number = verb-phrase.number) (actor . noun-phrase.sems) (action . verb-phrase.action) (object . verb-phrase.object) )

33. semantics - results (parse 'sentence '(the dogs chases a cat)) complete-edge 0 5 s1 sentence (the dogs...) nil Glitches: (numeric-agreement) s1 sentence -> (noun-phrase verb-phrase) Syntax (sentence (noun-phrase (det the) (noun dogs)) (verb-phrase (verb chases) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor specific canine) (action . hunts) (object any feline))

34. example 2 - lexicon (a det any ) (cat noun feline ) (chase verb hunts ) (dog noun canine ) (the det specific) (black adj (color black)) (large adj (size 7/10)) (small adj (size 3/10))

35. example 2 - grammar (build-grammar '((np (noun-phrase -> ?det *adj noun) (if det (quantification . det) (quantification undefined)) (qualifiers . *.adj) (object . noun) )) ))

36. example 2 - results (parse 'noun-phrase '(small black dog)) complete-edge 0 3 np noun-phrase (small...) nil np noun-phrase -> (?det *adj noun) Syntax (noun-phrase (adj small) (adj black) (noun dog)) Semantics (noun-phrase (quantification undefined) (qualifiers ((size . 3/10)) ((color . black))) (object canine))

37. example 2 - results small dogs chase the small cats and large dogs chase the large cats (sentence conjunction ((actor (quant undefined) (qual (size . 3/10)) (object . canine)) (action . hunts) (object (quant . specific) (qual (size . 3/10)) (object . feline))) ((actor (quant undefined) (qual (size . 7/10)) (object . canine)) (action . hunts) (object (quant . specific) (qual (size . 7/10)) (object . feline))))

38. semantic processing (one approach) • semantic rules in grammar  1st stage case frame • verb form  primitive action case frame • disambiguate & fill additional case frame slots • check references with world and/or dialog • do statement level inference • integrate with dialog • do event sequence dialog

39. step-1: produce raw case frame • verb cases the cat chased the rat in the kitchen the cat chased the rat into the kitchen • common cases source start-time instrument destination end-time beneficiary location duration

40. the ambiguity problem eg: the boy kicked the ball under the tree grammar rules S  S PP S  NP VP NP  ?det *adj noun NP  NP PP

41. example frame #1 actor (quant specific) (tags animate male human) (qual (age (range 3 13))) (root boy) action (root kick) object (root ball) (tags manip) (posn-relative (locator beneath) (object (root tree) ...etc... )

42. example frame #2 actor (quant specific) (tags animate male human) (qual (age (range 3 13))) (root boy) action (root kick) object (root ball) (tags manip) dest (posn-relative (locator beneath) (object (root tree) ...etc... )

43. example verb form #1 primitive strike prohibited object (tags manip) slots instrument (part-of $actor foot) legal start-time, end-time, duration instrument, beneficiary, location illegal source, dest

44. example verb form #2 primitive push required object (tags manip) slots instrument (part-of $actor foot) legal source, dest, start-time, end-time, instr, beneficiary, locatn, duration

45. semantic processing (one approach) • semantic rules in grammar  1st stage case frame • verb form  primitive action case frame • disambiguate & fill additional case frame slots • check references with world and/or dialog • do statement level inference • integrate with dialog • do event sequence dialog

46. integration with dialog dialogs have... • players (actors) • props (objects) • locations (from case frames) • themes (derived) • event sequences (from themes) • plans (from themes and/or derived)

47. event sequence set of... • players (actors) • props (objects) series of... • semantically encoded activities (matched) • escapes, exceptions & alternatives

48. reading – grammars, etc  A good source of links & references... “Computational Analysis of Prepositions” http://knol.google.com/k/abdul-baqi-sharaf/computational-analysis-of-prepositions/3hc3uny2z7r41/4# if you only plan to read one article... Baldwin, T. Kordoni, V and Villavicencio, A. 2009. Prepositions in Applications: A Survey and Introduction to the Special Issue ". Computational Linguistics 35 (2): 119–149. also... Litkowski, Kenneth C. and Orin Hargraves. 2007. SemEval-2007 task 06: Word-sense disambiguation of prepositions. In Proceedings of the 4th International Workshop on Semantic Evaluations, pages 24–29, Prague. Disambiguation of Preposition Sense Using Linguistically Motivated Features, Stephen Tratz and Dirk Hovy. Proceedings of the NAACL HLT Student Research Workshop and Doctoral Consortium, pages 96–100, Boulder, Colorado, June 2009. c 2009 Association for Computational Linguistics

49. reading – grammars, etc  the NLP dictionary: www.cse.unsw.edu.au/~billw/nlpdict.html for practical help with building grammars check the following (it is about 10 years old but then so is the English language :o) A Grammar Writer’s Cookbook. Miriam Butt, Tracy Holloway King, Marma-Eugenia Niño and Fridirique Segond also (for writing larger grammars) it is useful to find a book on grammar for tutors and/or students of English as a second language. for a broad (if a little formal) take on semantics try dipping into... Semantics-Oriented Natural Language Processing Mathematical Models and Algorithms. Vladimir Fomichov A. 2010

50. reading – kn rep for NLP logic and knowledge representation – a guide http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/9996/1/DSTO-TR-2324%20PR.pdf representing events for NLP http://www.google.co.uk/url?sa=t&rct=j&q=knowledge%20representation%20%22representing%20events%22&source=web&cd=6&sqi=2&ved=0CEgQFjAF&url=http%3A%2F%2Fwww.aaai.org%2Focs%2Findex.php%2FFSS%2FFSS10%2Fpaper%2Fdownload%2F2183%2F2819&ei=f6oWT_e7DeKC4gTMpaijBA&usg=AFQjCNFYmurwJR9oqfCRBimVprWRK45kew&cad=rja semantic networks & frames (2005) http://www.cs.bham.ac.uk/~jxb/IAI/w6.pdf VERL: An Ontology Framework for Video Events (2005) http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1524892