ICS 482 Natural Language Processing Words & Transducers-Morphology - 1

1. ICS 482Natural Language ProcessingWords & Transducers-Morphology - 1 Muhammed Al-Mulhem March 1, 2009

2. Steps of NLP Morphological Analysis Individual words are analyzed into their components Discourse Analysis Resolving references Between sentences Pragmatic Analysis To reinterpret what was said to what was actually meant Syntactic Analysis Linear sequences of words are transformed into structures that show how the words relate to each other Semantic Analysis A transformation is made from the input text to an internal representation that reflects the meaning

3. Morphology • Morphology: the study of the way words are built up from smaller meaning-bearing units, morphemes. • Morpheme: is the minimal meaning-bearing unit in a language • Example • fox: One morpheme fox. • cats: Two morphemes, cat and –s.

4. Morpheme Definitions • There are two broad classes of morphemes: • Stem • The main morpheme of the word, supplying the main meaning. • Affix • Add additional meaning of various kinds. It is further divided into: • Prefixes, • Suffixes, • Infixes, and • Circumfixes.

5. Morpheme Definitions Prefixes: Prefixes precede the stem. Suffixes: Suffixes follow the stem Circumfixes: Circumfixes do both Infixes: Infixes are inserted inside the stem. 5

6. Examples • Eats: composed of a stem eat and the suffix –s • Unbuckle: composed of a stem buckle and the prefix un- • English doesn’t really have circumfixes, but many other languages do. • In Germany, for example, the past participle of some verbs is formed by adding ge- to the beginning of the stem and –t to the end. • For example, the past participle of the verb sagen (to say) is gesagt (said).

7. Morpheme Definitions • A word can have more than one affix. • For example: the word rewrites has the prefix re-, the stem write, and the suffix –s. • The word unbelievably has the stem believe plus three affixes (un-, -able, and –ly). • English doesn’t use more than five affixes. • Other languages like Turkish can have words with ten affixes.

8. Morpheme Definitions There are many ways to combine morphemes to create words. Four of these are: Inflection Derivation Compounding Cliticization 8

9. Inflection • It is the combination of a word stem with a grammatical morpheme, usually resulting in a word of the same class as the original stem and usually filling some syntactic function like agreement. • For example, • English has inflectional morpheme –s for marking the plural on nouns. • The inflectional morpheme –ed for marking the past tense on verbs.

10. Derivation • It is the combination of a word stem with a grammatical morpheme, usually resulting in a word of a different class, often with a meaning hard to predict exactly. • For example, • The verb computerize can take the derivational suffix –ation to produce the noun computerization.

11. Compounding • It is the combination of a multiple word stems together. • For example, • The noun doghouse is the concatenation of the morpheme dog with the morpheme house.

12. Cliticization • It is the combination of a word stem with a clitic. • A clitic is a morpheme that acts syntactically like a word but is reduced in form and attached to another word. • For example, • The English morpheme ‘ve in the word I’ve is a clitic.

13. InflectionMorphology • English nouns have only two kinds of inflection: • an affix that marks plural and • an affix that marks possessive • Examples: Regular and irregular plurals.

14. InflectionMorphology • While the regular plural is spelled -s after most nouns, it is spelled -es after words ending in: • -s (ibis/ibises), • -z (waltz/waltzes), • -sh (thrush/thrushes), • -ch (finch/finches), and sometimes • -x (box/boxes). • Nouns ending in -y preceded by a consonant change the -y to -i (butterfly/butterflies).

15. InflectionMorphology • The possessive suffix is realized by: • Apostrophe + -s for regular singular nouns (llama’s) and plural nouns not ending in -s (children’s) and • A lone apostrophe after regular plural nouns (llamas’) and some names ending in -s or -z (Euripides’comedies).

16. InflectionMorphology • English has three kinds of verbs; • Main verbs, (eat, sleep, impeach), • Modal verbs (can,will, should), and • Primary verbs (be, have, do) • We will mostly be concerned with the main and primary verbs, because it is these that have inflectional endings.

17. InflectionMorphology • These regular verbs (e.g. walk, or inspect) have four morphological forms, as follow:

18. InflectionMorphology • Irregular verbs in English can have as many as eight or as few as three forms

19. Derivational Morphology • A common kind of derivation in English is the formation of new nouns from verbs or adectives (Nominalization) • Adjectives can also be derived from nouns and verbs

20. Cliticization Morphology English clitics include these auxiliary verbal form: Clitics is ambiguous. Example: She’s can mean she is or she has. 20

21. Compounding Morphology The kind of compound morphology we have discussed so far, in which a word is composed of a string of morphemes concatenated together is often called concatenative morphology. A number of languages have extensive non-concatenative morphology, in which morphemes are combined in more complex ways. Another kind of non-concatenative morphology is called templatic morphology or root-and-pattern morphology. Example: Read Chapter 3. 21

22. Aggrement We say that the subject noun and the main verb in English have to agree in number, meaning that the two must either be both singular or both plural. There are other kinds of agreement processes. For example nouns, adjectives, and sometimes verbs in many languages are marked for gender. A gender is a kind of equivalence class that is used by the language to categorize the nouns; each noun falls into one class. 22

23. Aggrement Many languages (for example Romance languages like French, Spanish, or Italian) have 2 genders, which are referred to as masculine and feminine. Other languages (like most Germanic and Slavic languages) have three (masculine, feminine, neuter). Gender is sometimes marked explicitly on a noun; for example Spanish masculine words often end in -o and feminine words in -a. 23

24. Aggrement But in many cases the gender is not marked in the letters or phones of the noun itself. Instead, it is a property of the word that must be stored in a lexicon as in the table bellow: 24

25. Parsing • Taking a surface input and identifying its components and underlying structure • Morphological parsing: parsing a word into stem and affixes and identifying the parts and their relationships • Stem and features: • goose goose +N +SGor goose + V • geese  goose +N +PL • gooses  goose +V +3SG • Bracketing: indecipherable [in [[de [cipher]] able]]

26. Why parse words? • For spell-checking • Is muncheble a legal word? • To identify a word’s part-of-speech (POS) • For sentence parsing, for machine translation, … • To identify a word’s stem • For information retrieval • Why not just list all word forms in a lexicon?

27. What do we need to build a morphological parser? • Lexicon: stems and affixes (w/ corresponding Part of Speech (POS)) • Morphotactics of the language: model of how morphemes can be affixed to a stem • Orthographic rules: spelling modifications that occur when affixation occurs • in  ilin context ofl(in- + legal)

28. Syntax and Morphology • Phrase-level agreement • Subject-Verb • Ali studies hard (STUDY+3SG) • Sub-word phrasal structures • ولحاجاتنا • و+لـ+حاجات+نا • and+for+need+PL+Poss:1PL • And for our needs

29. Morphotactic Models • English nominal inflection plural (-s) reg-n q0 q1 q2 irreg-pl-n • reg-n: regular noun • irreg-pl-n: irregular plural noun • irreg-sg-n: irregular singular noun irreg-sg-n • Inputs: cats, goose, geese

30. q1 q2 q0 adj-root1 -er, -ly, -est • Derivational morphology: adjective fragment un- adj-root1 q5 q3 q4  -er, -est adj-root2 • Adj-root1: clear, happy, real • Adj-root2: big, red

31. Using FSAs to Represent the Lexicon and Do Morphological Recognition • Lexicon: We can expand each non-terminal in our NFSA into each stem in its class (e.g. adj_root2 = {big, red}) and expand each such stem to the letters it includes (e.g. red  r e d, big  b i g) e r  q1 q2 q3 q7 q0 b d q4 -er, -est q5 i g q6

32. Limitations • To cover all of English will require very large FSAs with consequent search problems • Adding new items to the lexicon means re-computing the FSA • Non-determinism • FSAs can only tell us whether a word is in the language or not – what if we want to know more? • What is the stem? • What are the affixes? • We used this information to build our FSA: can we get it back?

33. Parsing with Finite State Transducers • cats cat +N +PL • Kimmo Koskenniemi’s two-level morphology • Words represented as correspondences betweenlexical level (the morphemes) and surface level (the orthographic word) • Morphological parsing :building mappings between the lexical and surface levels

34. Finite State Transducers • FSTs map between one set of symbols and another using an FSA whose alphabet  is composed of pairs of symbols from input and outputalphabets • In general, FSTs can be used for • Translator (Hello:مرحبا) • Parser/generator (Hello:How may I help you?) • To map between the lexical and surface levels of Kimmo’s 2-level morphology

35. FST is a 5-tuple consisting of • Q: set of states {q0,q1,q2,q3,q4} • : an alphabet of complex symbols, each is an i/o pair such that i  I (an input alphabet) and o  O (an output alphabet) and  is in I x O • q0: a start state • F: a set of final states in Q {q4} • (q,i:o): a transition function mapping Q x  to Q • Emphatic Sheep  Quizzical Cow a:o b:m a:o a:o !:? q0 q1 q2 q3 q4

36. FST for a 2-level Lexicon • Example c a t q3 q0 q1 q2 q5 q0 q1 q2 q3 q4 g e:o e:o s e

37. FST for English Nominal Inflection +N: reg-n +PL:^s# q1 q4 +SG:-# +N: irreg-n-sg q0 q2 q5 q7 +SG:-# irreg-n-pl q3 q6 +PL:-s# +N: Combining (cascade or composition) this FSA with FSAs for each noun type replaces e.g. reg-n with every regular noun representation in the lexicon

38. Orthographic Rules and FSTs • Define additional FSTs to implement rules such as consonant doubling (beg begging), ‘e’ deletion (make  making), ‘e’ insertion (watch  watches), etc.

39. Note: These FSTs can be used for generation as well as recognition by simply exchanging the input and output alphabets (e.g. ^s#:+PL)

40. Administration • Next Sunday: Quiz 1: 20 Minutes In the class • Assignment 2: What was your findings about Python? • New Assignment (3)

41. Assignment 3: Part 1A genre for your Corpora • Choose a Domain for your Corpora • Technology and Computers • Management • Weather • Sport • Economics • Politics • Education • Health care • History • Traditional Poems • New Poems • Other suggested fields

42. Assignment 3: Part 1A genre for your Corpora • Put your choice on the discussion list named 'My Corpora'. • read other selections before • Avoid selecting a topic that has been selected • You might need to suggest unlisted field • with the arrangement of the instructor • Collect text files and keep them in one directory as your corpora for future use • Suggested total size (sum of sizes of all text files) • larger than 10Mbyte of Arabic text

43. Assignment 3: Part 2List text files in a chosen directory • Write a program that allows the user to browse and select a directory, then the program will list the names of the text files in that directory. This program is needed to be used for future assignments and the course project. You can use any language you are mastering. However, Python might be a good choice

44. Assignment 3: Part 3 The most used n words & their frequencies in your corpora • After building your corpora, you need to find the most used 100 words in your corpora with their frequencies. You might do that by writing a program that let the user choose the directory of the corpora where the text files are located and find the most use n words. Where n could be 100.

45. Deliverables • A Report that contains the following headings: • Problem faced • Suggested Enhancements           • Test Cases and Screen Shots • How to compile and run the source code • Conclusion • Recommendation • Introduction • Description & Specification • Corpora (genre) and the reason • Design: High Level architecture • Implementation Issues • Accomplished Parts • Unaccomplished Parts • The source code of the program and the executable version of the same code. Please do not include the source code in the report.

46. Thank you السلام عليكم ورحمة الله