Pushpak Bhattacharyya CSE Dept., IIT Bombay 14 th April, 2011

1. CS460/626 : Natural Language Processing/Speech, NLP and the Web(Lecture 38–Universal Networking Language) Pushpak BhattacharyyaCSE Dept., IIT Bombay 14thApril, 2011

2. A Perpective Discourse Pragmatics Semantics Syntax Lexicon Morphology

3. UNL: a United Nations project • Started in 1996 • 10 year program • 15 research groups across continents • First goal: generators • Next goal: analysers (needs solving various ambiguity problems) • Current active language groups • UNL_French (GETA-CLIPS, IMAG) • UNL_English+Hindi • UNL_Italian (Univ. of Pisa) • UNL_Portugese (Univ of Sao Paolo, Brazil) • UNL_Russian (Institute of Linguistics, Moscow) • UNL_Spanish (UPM, Madrid)

4. World-wide Universal Networking Language (UNL) Project Marathi • Language independent meaning representation. English Russian UNL Spanish Japanese Hindi Others

5. Foundations and Applications • UNL Foundations • Semantic Relations • Universal Words • Attributes • How to write UNL expressions • UNL Applications • Machine Translation: Rule based and Statistical • Search • Text Entailment • Sentiment Analysis

6. UNL represents knowledge: John eats rice with a spoon Universal words Semantic relations attributes Repository of 42 Semantic Relations and 84 attribute labels

7. Sentence embeddings Deepa claimed that she had composed a poem. [UNL] agt(claim.@entry.@past, Deepa) obj(claim.@entry.@past, :01) agt:01(compose.@past.@entry.@complete, she) obj:01(compose.@past.@entry.@complete, poem.@indef) [\UNL]

8. English sentences: basic structure verb • A <verb> B • John eats bread • agt(eat.@entry, John) • obj(eat.@entry, bread) • A <verb> • John sleeps • aoj(sleep.@entry, John) • A <be> B • John is good • aoj(good.@entry, John) R2 R1 A B R2 verb R1 A B aoj A

9. Hindi sentences: basic structure verb • A B <verb> • John rotikhaataahai • agt(eat.@entry, John) • obj(eat.@entry, bread) • A <verb> • John sotaahai • aoj(sleep.@entry, John) • A <be> B • John acchaahai • aoj(good.@entry, John) R2 R1 A B R2 verb R1 A B aoj A

10. Complex English sentences: Use recursion on the basic structure eat A <verb> B • John who is a good boy eats bread which is toasted • agt(eat.@entry, :01) • obj(eat.@entry, :02) • aoj:01(boy, John.@entry) • mod:01(boy, good) • obj:01(toast, bread.@entry.@focus) agt obj :01 :02 :02 :01 toast boy aoj mod obj John good Bread Red arrows indicate entry nodes

11. Constituents of Universal Networking Language • Universal Words (UWs) • Relations • Attributes • Knowledge Base

12. What is a Universal Word (UW)? • Words of UNL • Constitute the UNL vocabulary, the syntactic-semantic units to form UNL expressions • A UW represents a concept • Basic UW (an English word/compound word/phrase with no restrictions or Constraint List) • Restricted UW (with a Constraint List ) • Examples: “crane(icl>device)” “crane(icl>bird)”

13. TheLexicon Format of the dictionary entry e.g., [minister] {}“minister(icl>person)”(N,ANIMT,PHSCL,PRSN); • Head word • Universal word • Attributes • Morphological - Pl(plural), V_ed(past tense form) • Syntactic - V(verb),VOA(verb of action) • Semantic - ANIMT(animate), PLACE, TIME [headword] {}“Universal word“(Attribute list);

14. TheLexicon (cntd) He forwarded the mail to the minister. Content words: [forward] {} “forward(icl>send)” (V,VOA) <E,0,0>; [mail] {}“mail(icl>message)”(N,PHSCL,INANI) <E,0,0>; [minister] {}“minister(icl>person)” (N,ANIMT,PHSCL,PRSN) <E,0,0>; Headword Universal Word Attributes

15. TheLexicon (cntd) He forwarded the mail tothe minister. function words: [he] {} “he” (PRON,SUB,SING,3RD) [the] {} “the” (ART,THE) <E,0,0>; [to] {} “to” (PRE,#TO) <E,0,0>; Headword Universal Word Attributes

16. Multilingual dictionary सार्वभौम शब्द मुख्य शब्द गुण farmer farmer(icl>creator) E N,ANIMT,FAUNA,MML,PRSN शेतकरी M N,M,ANIMT,FAUNA,MML,PRSN किसान H N,M,ANIMT,FAUNA,MML,PRSN,Na

17. The Features of a UW • Every concept existing in any language must correspond to a UW • The constraint list should beas small as necessary to disambiguate the headword • Every UW should be defined in the UNL Knowledge-Base (now wordnet)

18. Restricted UWs • Examples • He will hold office until the spring of next year. • The spring was broken. • Restricted UWs, which are Headwords with a constraint list, for example: “spring(icl>season)” “spring(icl>device)” “spring(icl>jump)” “spring(icl>fountain)”

19. How to create UWs? • Pick up a concept • the concept of “crane" as "a device for lifting heavy loads” or as “a long-legged bird that wade in water in search of food” • Choose an English word for the concept. • In the case for “crane", since it is a word of English, the corresponding word should be ‘crane' • Choose a constraint list for the word. • [ ] ‘crane(icl>device)' • [ ] ‘crane(icl>bird)'

20. Example: Hindi word ghar • ghar- house • usnegarmii me gharkiimarammatkii • he renovated the house in the summer • ghar- home • office kebaadgharlouto • return home after office • Ghar- family • bade gharkiibetii • girl from a renowned family

21. Example: ghar (cntd) • ghar- own country • bahutsaalbidesh me kaamkarkegharloutaaayaa • returned home after working abroad for many years • Ghar- astrological position • ashtamghar par budhhai • Mercury in in the eighth house

22. House in English Wordnet • 1. (1029) house -- (a dwelling that serves as living quarters for one or more families; "he has a house on Cape Cod"; "she felt she had to get out of the house") • 3. (51) house -- (a building in which something is sheltered or located; "they had a large carriage house") • 4. (39) family, household, house, home, menage -- (a social unit living together; "he moved his family to Virginia"; "It was a good Christian household“;)

23. House in English Wordnet • 7. (13) house -- (aristocratic family line; "the House of York") • 11. sign of the zodiac, star sign, sign, mansion, house, planetary house -- ((astrology) one of 12 equal areas into which the zodiac is divided)

24. Unambiguous construction of UWs Use constraints: Ontological, Semantic and Argument Example: forward a mail to the minister forward(icl>do, icl>send, agt>thing(icl>animate), obj>thing(icl>inanimate), gol>thing) Constraint types: icl>do: ontological, icl>send: semantic agt>thing, obj>thing, gol>thing: argument

25. UNL Relations

26. Relations constitute the syntax of UNL • Express how concepts (UWs) constitute a sentence • Represented as strings of 3 characters or less • A set of 41 relations specified in UNL (e.g., agt, aoj, ben, gol, obj, plc, src, tim,…) • Refer to a semantic role between two lexical items in a sentence

27. AGT / AOJ / OBJ • AGT  (Agent)Definition:  Agt defines a thing which initiates an action • AOJ (Thing with attribute)Definition:  Aoj defines a thing which is in a state or has an attribute • OBJ (Affected thing)Definition: Obj defines a thing in focus which is directly affected by an event or state

28. Examples • John broke the window. agt ( break.@entry.@past, John) • This flower is beautiful. aoj ( beautiful.@entry, flower) • He blamedJohn for the accident. obj ( blame.@entry.@past, John)

29. Example: UNL Graph with agt, obj, ben He carved a toy for the baby. carve(icl>cut) @ entry @ past agt ben obj he(iof>person) baby(icl>child) @def toy(icl>plaything)

30. GOL / SRC • GOL  (Goal : final state)Definition:  Gol defines the final state of an object or the thing finally associated with an object of an event • SRC  (Source : initial state)Definition:  Src defines the initial state of object or the thing initially associated with object of an event

31. deposit(icl>put) @ entry @ past gol agt obj account(icl>statement) money(icl>currency) mod mod I mod I bank(icl>possession) I GOL • I deposited my money in my bank account.

32. SRC They make a small income from fishing. make(icl>do) @ entry @ present src obj agt income(icl>gain) fishing(icl>business) they(icl>persons) mod small(aoj>thing)

33. PUR • PUR (Purpose or objective)Definition:  Pur defines thepurpose or objectives of the agent of an event or the purpose of a thing exist • This budget is for food. pur ( food.@entry, budget )mod ( budget, this )

34. RSN • RSN (Reason)Definition:  Rsn defines a reason why an event or a state happens • They selected him for his honesty. agt(select(icl>choose).@entry, they) obj(select(icl>choose) .@entry, he) rsn (select(icl>choose).@entry, honesty)

35. TIM • TIM (Time)Definition:  Tim defines the time an event occurs or a state is true • I wake up at noon. agt ( wake up.@entry, I )tim ( wake up.@entry, noon(icl>time))

36. PLC • PLC (Place)Definition:  Plc defines the place an event occurs or a state is true or a thing exists • Temples are veryfamous in India. aoj(famous.@entry, temple@pl)man(famous.@entry, very)plc(famous.@entry, India)

37. INS • INS   (Instrument) Definition:  Ins defines the instrument to carry out an event • I solved it with computer agt ( solve.@entry.@past, I )ins ( solve.@entry.@past, computer )obj ( solve.@entry.@past, it )

38. cover(icl>do) @ entry @ past agt ins obj John(iof>person) blanket(icl>object) baby(icl>child) @def INS John covered the baby with a blanket.

39. Attributes • Constitute syntax of UNL • Play the role of bridging the conceptual world and the real world in the UNL expressions • Show how and when the speaker views what is said and with what intention, feeling, and so on • Seven types: • Time with respect to the speaker • Aspects • Speaker’s view of reference • Speaker’s emphasis, focus, topic, etc. • Convention • Speaker’s attitudes • Speaker’s feelings and viewpoints

40. Tense: @past He went there yesterday • The past tense is normally expressed by @past {unl} agt(go.@entry.@past, he) … {/unl}

41. Aspects: @progress It’s raining hard. {unl} man ( rain.@entry.@present.@progress, hard ) {/unl}

42. Speaker’s view of reference • @def (Specific concept (already referred)) The house on the corner is for sale. • @indef (Non-specific class) There is a book on the desk • @not is always attached to the UW which is negated. He didn’t come. agt ( come.@entry.@past.@not, he )

43. Speaker’s emphasis • @emphasis John his name is. mod ( name, he )aoj ( John.@emphasis.@entry, name ) • @entry denotes the entry point or main UW of an UNL expression

44. How to generate UNL

45. sentence Word1 Word2 Word3 Word4 Wordn … windows LAW RCW RAW LCW Early Enco (1996-98) • Analysis windows -Two in number • Left Analysis Window (LAW) • Right Analysis Window (RAW) • Condition windows - Many in number • Left Condition Window (LCW) • Right Condition Window (RAW)

46. UNL Rule for a Semantic Relation ;Create relation between V and N2, after resolving the preposition preceding N2 <{V,VOA,:::}{N,TIME,DAY,ONRES,PRERES::tim:}P25; IF the left analysis window is on a verb(V) which is verb of action (VOA) AND the right analysis window is on a noun (N) and hasTIME, DAYattributefor which the preceding preposition (on) has been processedand deleted THEN set up the timrelation between V and N2. (indicated by < at the start of the rule)

47. UNL generation using NLP tools and resources

48. SRS based system

49. Multi parser based system

50. Evaluation • Recall = #expressions matched in gold and generated UNL #expressions expected in gold UNL • Precision = #expressions matched in gold and generated UNL #expressions in generated UNL • F1 score = 2 * recall * precision recall + precision