Human Language Technology

1. Human Language Technology Overview HLT

2. Acknowledgement • Material for some of these slides taken from • J Nivre, University of Gotheborg, Sweden • D. Jurafsky & J. Martin HLT

3. Human Language Technology HLT sometimes referred to as • Natural Language Processing • focus on linguistic processing • Computational Linguistics • focus on understanding language • Language Engineering • focus on practical tasks and results HLT

4. HLT – Engineering v. Science • Engineering • NLP is concerned with the design and implementation of effective NL input and output components for computational systems (Robert Dale 2000) • Science • The use of computers for linguistic research and applications HLT

5. HLT is Interdisciplinary • Linguistics • Theoretical • Applied • Computer Science • Algorithms • Compiling Techniques • Artificial Intelligence • Understanding, reasoning • Intelligent Action HLT

6. HLT is Commercial • Lot’s of exciting stuff going on… Powerset HLT

7. Google Translate HLT

8. Google Translate HLT

9. Web Q/A HLT

10. Web Analytics • Data-mining of social media • weblogs, discussion forums, message boards, user groups, and other forms of user generated media • Sentiment analysis, social network analysis • Product marketing information • Opinion tracking over space and time • Social network analysis • Buzz analysis (what’s hot, what topics are people talking about right now). HLT

11. HLT can help with • Understanding how language works • by implementing complex theories directly • More Natural Communication • development of multimodal M/M communication: language, speech, gesture • Development of multilingual applications • Knowledge Management • Language is the fabric of the web HLT

12. Language Enabled Applications • What makes an application a language processing application (as opposed to any other piece of software)? • An application that requires the use of knowledge about human languages • Example: Is Unix wc (word count) an example of a language processing application? HLT

13. Language Enabled Applications • Word count? • When it counts words: Yes • To count words you need to know what a word is. That’s knowledge of language. • When it counts lines and bytes: No • Lines and bytes are computer artifacts, not linguistic entities HLT

14. Small Spelling correction Hyphenation Medium Word-sense disambiguation Named entity recognition Information retrieval Big Question answering Conversational agents Automatic Summarisation Machine translation Stand-alone Enabling applications Funding/Business plans Topics: Applications HLT

15. Big Applications • These kinds of applications require a tremendous amount of knowledge of language. • Consider the following interaction with HAL the computer from 2001: A Space Odyssey HLT

16. HAL from 2001 • Dave: Open the pod bay doors, Hal. • HAL: I’m sorry Dave, I’m afraid I can’t do that. • http://www.youtube.com/watch?v=kkyUMmNl4hk HLT

17. What’s needed? • Speech recognition and synthesis • Knowledge of the English words involved • What they mean • How groups of words fit together into groups • What the groups mean • How the groups relate to each other. HLT

18. What’s needed? • Dialog • It is polite to respond, even if you’re planning to kill someone. • It is polite to pretend to want to be cooperative (I’m afraid, I can’t…) HLT

19. Summary of Application Areas • Document Processing • Classification • Summarisation • Information Extraction • Question Answering • Information Retrieval • Dialogue • Multilinguality • Machine Translation • Translation tools • Multimodality • speech • intonation • image HLT

20. Basic Problems • Analysis • Conversion of NL input to internal representations • Generation • Conversion of internal representations to NL output • Issues • What kind of input/output/representations? • Role of learning • Supervised v unsupervised • What training data is available? • System Evaluation HLT

21. Levels of Linguistic Knowledge • Phonetics/Phonology: sound structure • Morphology: word structure • Syntax: sentence structure • Semantics: meanings • Pragmatics: use of language in context • Discourse: paragraphs, texts, dialogues HLT

22. Each level of knowledge is associated with an encapsulated set of processes. Interfaces are defined that allow the various levels to communicate. This often leads to a pipeline architecture. Processing Pipelines HLT

23. Ambiguity • Computational linguists are obsessed with ambiguity • Ambiguity is a fundamental problem of computational linguistics • Resolving ambiguity is a crucial goal • Ambiguity arises at different levels of analysis HLT

24. Ambiguity – different flavours • LexicalI made her duck • SyntacticYoung men and women • ReferentialShe did it • PragmaticCan you pass the salt? HLT

25. Ambiguity • Find at least 5 meanings of this sentence: • I made her duck • I cooked waterfowl for her benefit (to eat) • I cooked waterfowl belonging to her • I created the (plaster?) duck she owns • I caused her to quickly lower her head or body • I waved my magic wand and turned her into undifferentiated waterfowl HLT

26. Ambiguity is Pervasive I made her duck • I caused her to quickly lower her head or body • Lexical category: “duck” can be a N or V • I cooked waterfowl belonging to her. • Lexical category: “her” can be a possessive (“of her”) or dative (“for her”) pronoun • I made the (plaster) duck statue she owns • Lexical semantics: “make” can mean “create” or “cook” HLT

27. Ambiguity is Pervasive • Grammar: Make can be: • Transitive: (verb has a noun direct object) • I cooked [waterfowl belonging to her] • Ditransitive: (verb has 2 noun objects) • I made [her] (into) [undifferentiated waterfowl] • Action-transitive (verb has a direct object and another verb) • I caused [her] [to move her body] HLT

28. Ambiguity is Pervasive • Phonetics! • I mate or duck • I’m eight or duck • Eye maid; her duck • Aye mate, her duck • I maid her duck • I’m aid her duck • I mate her duck • I’m ate her duck • I’m ate or duck • I mate or duck HLT

29. Dealing with Ambiguity • Four possible approaches: • Tightly coupled interaction among processing levels; knowledge from other levels can help decide among choices at ambiguous levels. • Pipeline processing that ignores ambiguity as it occurs and hopes that other levels can eliminate incorrect structures. HLT

30. Dealing with Ambiguity • Probabilistic approaches based on making the most likely choices • Don’t do anything, maybe it won’t matter • We’ll leave when the duck is ready to eat. • The duck is ready to eat now. • Does the “duck” ambiguity matter with respect to whether we can leave? HLT

31. Ways of Studying NLP • By ApplicationMT, IE, IR etc. • By Approachrational vs. empirical • By Linguistic Levelmorphology, syntax etc. • By Algorithm HLT

32. Algorithms • State Machines • automata and transducers • Rule Systems • regular and context free grammars • Search • top-down/bottom-up parsing • Probabilistic algorithms HLT

33. Organisation of Course • Module 1: Words • Linguistics: Morphological Structure • Morphological Processing • LAB + Assignment I • Module 2: Sentences • Linguistics: Syntactic Structure • NL Parsing Algorithms • LAB + Assignment II • Module 3: Texts • Statistics • Text Classification • LAB + Assignment III HLT

34. Course Information • Course Websitehttp://staff.um.edu.mt/mros1/hlt • Reference Texts • D. Jurafsky and J. Martin, Speech and Language Processing, 2nd Edition, Prentice-Hall • S. Bird, E. Klein and E. Loper, Natural Language Processing with Pythonhttp://www.nltk.org • Thank you HLT