NL-Soar tutorial

1. NL-Soar tutorial Deryle Lonsdale and Mike Manookin Soar Workshop 2003

2. Acknowledgements • The Soar research community • The CMU NL-Soar research group • The BYU NL-Soar research group humanities.byu.edu/nlsoar/homepage.html

3. Tutorial purpose/goals • Present the system and necessary background • Discuss applications (past, present and possible future) • Show how the system works • Dialogue about how best to disseminate/support the system

4. What is NL-Soar? • Soar-based cognitive modeling system • Natural-language focus: comprehension, production, learning • Used specifically to model language tasks: acquisition, translation, simultaneous inter-pretation, parsing difficulties, etc. • Also used to integrate language performance with other modeled tasks

5. How we use language • Speech • Language acquisition • Reading • Listening • Monolingual/bilingual language • Discourse/conversational settings

6. Why model language? • Can be insightful into properties of language • Understand interplay between language and other cognitive processes (memory, attention, tasks, etc.) • Has NLP applications

7. Language modeling • Concise, modular formalisms for language processing • Language: learning, situated use • Rules, lexicon, parsing, deficits, error production, task interference, etc. • Machine learning, cognitive strategies, etc. • Various architectures: TiMBL, Ripper, SNoW • Very active research area; theory + practice • Various applications: bitext, speech, MT, IE

8. How to model language • Statistical/probabilistic • Hidden Markov Models • Cognition-based • NL-Soar • ACT-R • Non-rule-based • Analogical Modeling • Genetic algorithms • Neural nets

9. The larger context: UTC’s (Newell ’90) • Develop a general theory of the mind in terms of a single system (unified model) • Cognition: language, action, performance • Encompass all human cognitive capabilities • Observable mechanisms, time course of behaviors, deliberation • Knowledge levels and their use • Synthesize and apply cognition studies • Match theory with experim. psych. results • Instantiate model as a computational system

10. From Soar to NL-Soar • Unified theory of cognition+ • Cognitive modeling system+ • Language-related components • Unified framework for overall cognition including natural language (NL-Soar)

11. A little bit of history (1) • UTC doesn’t address language directly: • “Language should be approached with caution and circumspection. A unified theory of cognition must deal with it, but I will take it as something to be approached later rather than sooner.” (Newell 1990, p.16)

12. A little bit of history (2) • CMU group starts NL-Soar work • Rick Lewis dissertation on parsing (syntax) • Semantics, discourse enhancements • Generation • Release in 1997 (Soar 7.0.4, Tcl 7.x) • TACAIR integration • Subsequent work at BYU

13. NL-Soar applications • Parsing breakdown • NTD-Soar (shuttle pilot test director) • TacAir-Soar (fighter pilots) • ESL-Soar (language acquisition: Polish speakers learning English) • SI-Soar (simultaneous interpretation: EnglishFrench) • AML-Soar (Analogical Modeling of Language) • WNet/NL-Soar (WordNet integration)

14. An IFOR pilot (Soar+NL-Soar)

15. NL-Soar processing modalities • Comprehension (NLC): parsing, semantic interpretation (wordsstructures) • Discourse (NLD): track how conversation unfolds • Generation (NLG): realize a set of related concepts verbally • Mapping: converting from one semantic representation to another • Integration with other tasks

16. From pilot-speak to language • 1997 release’s vocabulary was very limited • Lexical productions were hand-coded as sp’s (several very complex sp’s per lexical item) • Needed a more systematic, principled way to represent lexical information • WordNet was the answer

17. Before: Severely limited, ad-hoc vocabulary No morphological processing No systematic knowledge of syntactic properties Only gross semantic categorizations After: Wide-coverage English vocabulary A morphological interface (Morphy) Subcategorization information Word senses and lexical concept hierarchy Integration with WordNet

18. What is WordNet? • Lexical database with wide range of information • Developed by Princeton CogSci lab • Freely distributed • Widely used in NLP, ML applications • Command line interface, web, data files • www.princeton.cogsci.edu/~wn

19. WordNet as a lexicon • Wide-coverage English dictionary • Extensive lexical, concept (word sense) inventory • Syncategorematic information (frames etc.) • Principled organization • Hierarchical relations with links between concepts • Different structures for different parts of speech • Hand-checked for reliability • Utility • Designed to be used with other systems • Machine-readable database • Used as a base/standard by most NLP researchers

20. Hierarchical lexical relations • Hypernymy, hyponymy • Animal  dog  beagle • Dog is a hyponym (specialization) of the concept animal • Animal is a hypernym (generalization) of the concept dog • Meronymy • Carburetor <--> engine <--> vehicle

21. Hierarchical relationships dog, domestic dog, Canis familiaris -- (a member of the genus Canis (probably descended from the common wolf) that has been domesticated by man since prehistoric times; occurs in many breeds; "the dog => canine, canid -- (any of various fissiped mammals with nonretractile claws and typically long muzzl => carnivore -- (terrestrial or aquatic flesh-eating mammal; terrestrial carnivores have four or five clawed digits on each limb) => placental, placental mammal, eutherian, eutherian mammal -- (mammals having a placenta; all mammals except monotremes and marsupials) => mammal -- (any warm-blooded vertebrate having the skin more or less covered with hair; young are born alive except for the small subclass of monotremes) => vertebrate, craniate -- (animals having a bony or cartilaginous skeleton with a segmented spinal column and a large brain enclosed in a skull or cranium) => chordate -- (any animal of the phylum Chordata having a notochord or spinal column) => animal, animate being, beast, brute, creature, fauna -- (a living organism characterized by voluntary movement) => organism, being -- (a living thing that has (or can develop) the ability to act or function independently) => living thing, animate thing -- (a living (or once living) entity) => object, physical object -- (a tangible and visible entity; an entity that can cast a shadow; "it was full of rackets, balls and other objects") => entity, physical thing -- (that which is perceived or known or inferred to have its own physical existence (living or nonliving)

22. Complexity Granularity Coverage Widely used Usable information Coverage WordNet coals / nuggets you’ll see...

23. head 30 line 29 point 24 cut 19 case 18 base 17 center 17 place 17 play 17 shot 17 stock 17 field 16 lead 16 pass 16 break 15 charge 15 form 15 light 15 position 15 roll 15 slip 15 break 63 make 48 give 45 run 42 cut 41 take 41 carry 38 get 37 hold 36 draw 33 fall 32 go 30 play 29 catch 28 raise 27 call 26 check 26 cover 26 charge 25 pass 25 clear 24 Sample WordNet ambiguity

24. Back to NL-Soar • Basic assumptions / approach • NLC: syntax and semantics (Mike) • NLD: Deryle • NLG: Deryle

25. Basic assumptions • Operators • Subgoaling • Learning/chunking

26. NL-Soar comprehension op’s • Lexical access • Retrieve from a lexicon all information about a word’s morpho/syntactic/semantic properties • Comprehension • Convert an incoming sentence into two representations • Utterance-model constructors: syntactic • Situation-model constructors: semantic

27. Sample NL-Soar operator types • Attach a subject to its predicate • Attach a preposition and its noun phrase object together • NTD: move eye, attend to message, acknowledge • IFOR: report bogey • Attach an action with its agent

28. A top-level NL-Soar operator

29. Subgoaling in NL-Soar (1)

30. Subgoaling in NL-Soar (2)

31. The basic learning process (1)

32. The basic learning process (2)

33. The basic learning process (3)

34. Lexical access processing • Performed on incoming words • Attended to from decay-prone phono buffer • Relevant properties retrieved • Morphological • Syntactic • Semantic • Basic syn/sem categories projected • Provides information for later syn/sem processing

35. Morphology in NL-Soar • Previous versions: fully inflected lexical entries via productions • Now: TSI code to interface directly with WordNet data structures • Morphy: subcomponent of WordNet to return baseform of any word • Had to do some post-hoc refinement

37. Comprehension

38. NL-Soar Comprehension • Lexical Access • Morphology • Syntax • Semantics Overview of topics:

39. How NL-Soar comprehends • Words are input into the system 1 at a time • The agent receives words in an input buffer • After a certain amount of time the words decay (disappear) if not attended to • Each word is processed in turn; “processed” means attended to (recognized, taken into working memory) and incorporated into relevant linguistic structures • Processing units: operators, decision cycles

40. NL-Soar comprehension op’s • Lexical access • retrieve from a lexicon all information about a word’s morpho/syntactic/semantic properties • Comprehension • convert an incoming sentence into two representations • Utterance-model constructors: syntactic • Situation-model constructors: semantic

41. Lexical Access • Word Insertion: Words are read into NL-Soar one at a time. • Lexical Access: After a word is read into NL-Soar, the word frame is accessed from WordNet. • WordNet: An online database that provides information about words such as their part of speech, morphology, subcategorization frame, and word senses.

42. Shared architecture • Exactly same infrastructure used for syntactic comprehension and generation • Syntactic u-model • Semantic s-model • Lexicon, lexical access operators • Syntactic u-cstr operators • Decay-prone buffers • Generation leverages comprehension • Learning can be bootstrapped across modalities

43. How much should an op do?

44. Memory & Attention • Word enter the system one at a time. • If a word is not processed quickly enough, then it decays from the buffer and is lost.

45. Assumptions • Interpretive Semantics (syntax is prior) • Yet there is some evidence that this is not the whole story • Other computational alternatives exist (tandem) • We hope to be able to relax this assumption eventually

46. Syntax

47. NL-Soar Syntax (overview) • Representing Syntax (parsing, X-bar) • Subcategorization & WordNet • Sample Sentences • U-cstrs (constraint checking) • Snips • Ambiguity

48. Linguistic models • Syntactic model: X-bar syntax, basic lexical properties (verb subcategorization, part-of-speech info, features, etc.) • Semantic model: lexical-conceptual structure (LCS) that is leveraged from the syntactic nodes and lexicon-based semantic properties • Assigner/receiver (A/R) sets: keep track of which constituents can combine with which other ones • I/O buffers

49. Syntactic phrases • One or more words that are “related” syntactically • Form a constituent • Have a head (most important part) • Have a category (derived from the head) • Have specific order, distribution, cooccurrence patterns (in English)

50. English parse tree are