A Short History of Two-Level Morphology

1. A Short History of Two-Level Morphology Lauri Karttunen, Xerox PARC Kenneth R. Beesley, XRCE

2. Overview • Introduction • What is morphology? • Two strains of finite-state morphology • State of the art circa 1980. • Two-Level Morphology • Origins, basic idea • Implementations, compilers • Recent Developments

3. What is Morphology? • Morphosyntax • Words are composed of smalled units of meaning called morphemes that must be combined in a certain order. • piti-less-ness vs. *piti-ness-less • Morphological Alternations • The shape of a morpheme depends on its environment. • piti-less vs *pity-less

4. Lexical form fst 1 Intermediate form fst 2 ... fst n Surface form Sequential Model Ordered sequence of rewrite rules (Chomsky & Halle ‘68) can be modeled by a cascade of finite-state transducers Johnson ‘72 Kaplan & Kay ‘81

5. Lexical form ... fst 2 fst 1 fst n Surface form Parallel Model Set of parallel of two-level rules compiled into finite-state automata interpreted as transducers Koskenniemi ‘83

6. Lexical form Lexical form fst 1 ... fst 2 fst 1 fst n Intermediate form fst 2 Surface form compose ... FST fst n Surface form Sequential vs. Parallel intersect Perhaps too large to be practical.

7. State of the Art circa 1980 • Cut-and-paste analysis • leaves --> leave --> leav --> leaf • ad-hoc programs, not reversible for generation • Paradigm tables • comprendre 45 • not reversible for analysis, impractical for morphologically complex languages • Chomsky-Halle rewrite rules • x -> y / z _ w • computationally complex, no implementation, reversible?

8. Discovery and Rediscovery • C. Douglas Johnson (1972) showed that • phonological rewrite rules are interpreted in a way that makes them less powerful than they appear • rewrite rules can be modeled by finite transducers • for any two finite transducers applied in a sequence there exists an equivalent single transducer (Schützenberger 1961). • Johnson’s result was ignored and forgotten, rediscovered by Ronald M. Kaplan and Martin Kay at Xerox around 1980.

9. k a N p a n k a m p a n k a m m a n Sequential Application N -> m / _ p p -> m / m _

10. N:m 2 k a N p a n k a m p a n k a m m a n p m N:m ? m 0 ? p 1 N N m p 1 ? m 0 ? p:m Sequential Application in Detail 0 0 0 2 0 0 0 0 0 0 1 0 0 0

11. N:m p:m 3 N:m m 0 m ? ? p p:m N:m 1 2 m N ? N N Composition k a N p a n k a m m a n 0 0 0 3 0 0 0

12. Building a Compiler • Requires a finite-state calculus • concatenation, union, intersection, complementation... • Constraints are regular languages • “if p occurs then q follows” • . . . p. . . . q. . . . • ?* p ?* q ?* • ~[ ?* p ~[ ?* q ?* ]] • The idea of double negation was Kaplan and Kay’s first insight. Many details remained to be worked out.

13. k a m p a n k a m m a n k a m p a n k a m m a n The Problem of “Overanalysis” k a N p a n

14. The Birth of Two-Level Morpholgy • In the spring of 1981 Kimmo Koskenniemi came to UT at Austin in search of a dissertation topic. • Karttunen demoed his TEXFIN analyzer/generator for Finnish. • Kaplan and Kay briefed him about their discoveries. Koskenniemi visited PARC. • After a gestation period of about a year, two-level morphology was born.

15. The Three Ideas of Two-Level Morphology • Rules are symbol-to-symbol constraints that are applied in parallel, not sequentially like rewrite rules. • The constraints can refer to the lexical context, to the surface context or to both contexts at the same time. • Lexical lookup and morphological analysis are performed in tandem.

16. k a N p a n k a m m a n k a N p a n k a m m a n Two-Level Constraints 1 N:m correspondence requires a following p on the lexical side. p:m correspondence requires a preceding m on the surface side. In this context, all other possible realization of a lexical p are prohibited. In this context, all other possible realization of a lexical N are prohibited.

17. s p y 0 + s s p i e 0 s s p y 0 + s s p i e 0 s y:i <=> _ 0:e 0:e <=> Cons: y: _ +: s: Two-Level Constraints 2

18. k a N N:m Rule p:m Rule k a m m a n Parallel Application p

19. k a p N N:m Rule p:m Rule k a m m a n Lookup and Analysis in Tandem

20. Two-Level Implementations • 1982 Koskenniemi (Pascal) • 1983 Karttunen et al. at UTexas (Lisp) • 1986- Antworth et al. at SIL (C) • 1987 Black et al. Alvey Project (Lisp) • 1989 Beesley Alpnet (Lisp) • 1991 Pulman et al. ALEP (Prolog) • 1995 Carter SRI CLE (Prolog) • 1995 Petitpierre et al. MULTEXT (C)

21. Two-Level Rule Compilers • 1985 Kaplan and Koskenniemi: the basic compilation algorithm developed during Koskenniemi’s visit at CSLI at Stanford on a Dandelion (Xerox Lisp machine). It was based on the techniques Kaplan and Kay had developed for compiling rewrite-rules. • 1985-87 Koskenniemi and Karttunen: the first compiler • 1992 Current C version (twolc) by Karttunen and Beesley. • 1996 Grimley-Evans, Kiraz, Pulman: compiler for a “partition-based” two-level formalism

22. Seeds of Dissatisfaction • Two-level morphological analyzers became a standard component in natural language processing systems. • But there was no publicly available compiler until recently. • Morphotactics was “improved” by adding feature unification.Two-level analyzers acquired a reputation for being slow. • Two-level rules are notoriously difficult to write, even with a compiler.

23. makun ma un General rule k:0 pukun puvun Vowel _ Vowel Exception k:v u _ u Resolution by underspecification: k:0 | k:v <=> Vowel _ Vowel k:v <=> u _ u Rule Conflicts

24. Recent Developments • The pioneers of finite-state morphology new that a cascade of transducers or a set of parallel rules could be combined into a single transducer. • But the resulting single transducer is typically huge compared to the size of the original rule networks. Impractical in most cases. • The obvious solution, not seen for a long time, was to compose the rules with the lexicon.

25. Source Lexicon Lexical Transducer composition o R1 R2 Rn & & ... Karttunen, Kaplan, Zaenen 1992 intersection canonical form inflection codes s p y 0 +Noun +PL s p i e 0 s inflected form Lexical Transducer

26. Cascade of Compositions Ci o composition Source Lexicon Lexical Transducer o replace rule R1 o Cj constraint o ... Rn

27. Linguistic Issues • The idea of rules as parallel constraints was not picked up by mainstream linguists in the 80’s. • Many arguments had been advanced to show that phonological alternations could not be described or explained without sequential rewrite rules. • The two-level model was perceived as a computational “hack”, not worthy of academic interest.

28. Rise of Optimality Theory • Optimality Theory, the dominant paradigm in phonology since 1993 is a two-level model with parallel constraints. • Most optimality constraints can be encoded trivially as two-level rules. • The main difference is that OT constraints are ranked and violable.

29. Lexical form Lexical form fst 1 ... fst 2 fst 1 fst n Intermediate form fst 2 Surface form ... fst n Surface form Back to the Big Picture While the sequential model was popular among mainstream linguists, computational linguists preferred the parallel model. Now it is almost the other way round, although for computational linguists there is no substantive difference.