Dependency Representations, Grammars, Folded Structures among Other Things!

1. Dependency Representations, Grammars, Folded StructuresamongOther Things! Aravind K Joshi University of Pennsylvania Philadelphia USA DEPLING, 2013 August 28 2013 Charles University, Prague, Czech Republic

2. Outline • How complex dependencies can be? - Projective vsNonprojective - Representations at different levels • Direct representations or via formal grammars • Empirical studies: What dependencies appear in annotated corpora and not just how often? • Dependencies as Folded Structures!!

3. Complexity of Dependencies • What do we know from the formal side? • Mildly Context Sensitive Languages (MCSL)-- includes Context Free Languages (CFL) -- Limited crossing dependencies -- Polynomiallyparsable

4. Complexity of Dependencies • Although TAG’s (TAL’s) are in MCSL, they are much more restricted! • Although TAG’s support crossing dependencies, these crossing dependencies are well-nested (nested dependencies of context-free languages are well-nested—Pumping Lemma!) Proposed in Joshi et al. 1985 but proved only in 2010 by Kanazwa!

5. Complexity of Dependencies • The language MIX, proposed by Bach (1982)as an Extreme Case of Scrambling! • Bach Language, MIXMIX = { w| for each n, w is a string containing n a’s, n b’s, and n c’s, in any order}Treating, each w as a ‘scrambled’ version ofn clauses, each clause containing one a, one b, and one c. • MIX is thus an extreme case of non-projectivity!

6. MIX • Joshi (1985) suggested that TAG’s and their variants introduced for linguisticdescriptions should not be able to generate MIX, thereby excluding it from the class of Mildly Context Sensitive Languages (MCSL) • Many attempts to prove this conjecture did not succeed, until very recently! • Kanazawa and Sylvati finally proved thisconjecture in 2012 (paper presented at ACL 2012)!

7. Complexity of Dependencies • Varieties of TAG grammars all weaklyequivalent to TAG but capable of providingstructural descriptions going beyond standard TAG -- Tree Local Multicomponent TAG (MCTAG) -- Very Limited use of Set Local MCTAG -- Adequate for Scrambling and Clitic Climbing constructions (Joint work with Joan Chen Main and Tonia Bleam, 2011, 2012)

8. SINGLE TREE or SETS of TREESrequiring skilled tree surgeries to force a single tree over a sentence • Parentheticals • Epithets • Displaced adjectives, PP’s, etc. • Right node raising • Extraposition from NP • Sentential relatives • Coordinations

9. One treecoveringthe whole sentence W1 W2 W3 W4 W5 W6 • Single tree rooted in one root node • Every word is covered • All connections between the nodes are in the same dimension

10. Parentheticals Mary, John thinks, will win the race Mary, John thinks, will win the race Arterial Roots John thinks is attached to the root node of the Mary will win the race treein an orthogonal dimension, reflecting the different semantic nature of thisattachment

11. Epithets I finished the damn book Arterial Roots I finished the damn book damn is attached to the root node of the I finished the book tree an orthogonaldimension, reflecting the different semantic nature of this attachment

12. More examples • Extraposition from NP: The gardener finally came, who had the keys • Misplaced adjectives: An occasional sailor walked by • Sentential relatives:John believes* Mary will finish her dissertation this year, which no one expected her to do * This example is from Bonnie Webber.

13. Some benefits for not insisting on a single tree • Not going for a single tree may ease the burden on the annotators • Sometimes syntax does more work than necessary! Very often at the discourse annotation stage some work done by syntax has to be undone. -- Syntax should have annotated the sentence with two chunks linked in an orthogonal dimension ** It is only at the discourse annotation stage the final decision of the attachment can be made -- Striking and very frequent examples of this situation arise inATTRIBUTION

14. S NP VP SBAR-ADV VP There IN S have been no Orders for the Cray-3 NP VP though S the company V it is talking With several prospects says Discourse arguments Syntactic arguments • There have been no orders for the Cray-3 so far, thoughthe company says it is talking with several prospects. • Discourse semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “there being a possibility of some prospects”. • Sentence semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “the company saying something”.

15. Attribution cannot always be excluded by default Advocates said the 90-cent-an-hour rise, to $4.25 an hour by April 1991, is too small for the working poor, whileopponents argued that the increase will still hurt small business and cost many thousands of jobs In this example, the attributing phrases stay with the argumentsof the connective while This decision can only be made at the discourse levelAt the sentence level two trees covering the sentence with the two trees connected in an orthogonal dimension would have been the best decision!

16. Annotation Overview: Attribution in WSJ 34% of discourse relations are attributed to an agent other than the writer.

17. Types of Dependencies • Word to Word John loves mangoes John bought the house Predicate argument relation?

18. Types of Dependencies Word to Phrase John bought the house Predicate argument relation?

19. Types of Dependencies Phrase to Word John took a walk

20. Types of Dependencies Phrase to Phrase The old man took a walk

21. Types ofDependencies How much of the phrase to be included in the argument? By convention (?) we take the maximal phrase. John bought [the house next door which was on sale for over a year] the house the house next door the house next door which was on sale for over a year What about the minimal phrase that is sufficient to identifythe referent in the context (discourse context, for example)?

22. S NP VP SBAR-ADV VP There IN S have been no Orders for the Cray-3 NP VP though S the company V it is talking With several prospects says Discourse arguments Syntactic arguments • There have been no orders for the Cray-3 so far, thoughthe company says it is talking with several prospects. • Discourse semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “there being a possibility of some prospects”. • Sentence semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “the company saying something”.

23. S SBAR-ADV NP-SBJ VP S IN MD VP the application by his RGH Inc. Although NP-SBJ VP could VB NP takeover experts VBD SBAR signal said his interest in helping revive a failed labor- management bid NP-SBJ VP VBD SBAR they Mr. Steinberg will make a bid by himself doubted • Althoughtakeover experts said they doubtedMr. Steinberg will make a bid by himself, the application by his Reliance Group Holdings Inc. could signal his interest in helping revive a failed labor-management bid. • Discourse semantics: contrary-to-expectation relation between “Mr. Steinberg not making a bid by himself” and “the RGH application signaling his bidding interest”. • Sentence semantics: contrary-to-expectation relation between “experts saying something” and “the RGH application signaling Mr. Steinberg’s bidding interest”.

24. Attribution cannot always be excluded by default • Advocates said the 90-cent-an-hour rise, to $4.25 an hour by April 1991, is too small for the working poor, whileopponents argued that the increase will still hurt small business and cost many thousands of jobs.

25. Do we want a single tree over a sentence? • There are many constructions in language that suggest that the single tree hypothesis may be wrong -- Parentheticals, supplements, sentential relatives, among others are problematic for the single tree hypothesis Mary, John thinks, will win the election (John thinks is attached to the S node medially but it has scope over Mary will win the election)

26. John heard thatMary finally finished her dissertation,which no one ever expected her to do so ( (1) John heard thatand (2) which no one ever expected her to do both have scope over (3) Mary finally finished her dissertation. Both (1) and (2) are attached to the root node S but neither (1) nor (2) have scope over the other)

27. Alternative Lexicalization(AltLex) A discourse relation is inferred between two sentences which do not contain an Explicit connective, but insertion of an Implicit connective leads to redundancy. This is because the relation is alternatively lexicalized by some non-connective expression: • Under a post-1987 crash reform, the Chicago Mercantile Exchange wouldn’t permit the December S&P futures to fall further than 12 points for a half hour. AltLex = (consequence)That caused a brief period of panic seeling of stocks on the Big Board.

28. AltLex expressions often do not correspond to syntactic constituencies. Under a post-1987 crash reform, the Chicago Mercantile Exchange wouldn’t permit the December S&P futures to fall further than 12 points for a half hour. AltLex = (consequence)That caused a brief period of panic selling of stocks on the Big Board. S NP-SBJ VP DT VBD DT PP-LOC That caused a brief period of panic selling…..

29. Syntactic Structures as Folded Structures Analogous to Secondary or Tertiary Structures of Biomolecules

30. Biological Sequences • DNA, RNA, PROTEIN Sequences -- DNA and RNA: sequences of four nucleotides -- A, C, G, and T or A, C, G, and U -- Matching Pairs: A, T(U) and C, G -- Proteins: Sequences of twenty amino acids

31. RNA secondary structure

32. RNA secondary structure: Pseudoknots

33. Dependencies as Folded Structures apples John has eaten John has eaten apples apples eaten John has

34. Dependencies as Folded Structures John has eat en apples John has eaten apples apples eat John has en

35. Subject Relatives The cat that chased the rat fled NP1 that V1 NP2 V2 V2 NP1 that V1 NP2

36. Object Relatives Goes into another plane and comes out The cat that the dog chased fled NP1 that NP2 V2 V1 NP2 V1 NP1 that V2 Object Relatives are more complex than Subject Relatives, even at the first level.

37. Crossing Crossing Versus Nested Dependencies Both (1) and (2) can be folded in one plane! (1’) NP1 NP2 V2 V1 (2’) NP1 NP2 NP3 V3 V2 V1 Nested (1) NP1 NP2 V1 V2 (2) NP1 NP2 NP3 V1 V2 V3 (1’)canbe folded in one plane. (2’)cannotbe folded in one plane. Beyond 2 levels of embedding this difference disappears!! cf Bach and Marslen Wilson (1985)

38. RNA secondary structure: Pseudoknots

39. Pseudoknots in Linguistic Structures N1 N2 N3 V3 V2 V1 Move N2 N3 V3 V2 to the right of V1 and then Move N2 N3 back N1 V1 N2 N3 V2 V3 N1 N2 N3 V1 V3 V2

40. Pseudoknots In Linguistic Structures N1 N2 N3 V1 V3 V2 Remnant Extraposition N1 V1 N2 N3 V2 V3 FOLDED STRUCTURE AS THE SYNTACTIC STRUCTURE!!

41. Folded Structures as Syntactic Structures When Subject Relatives and Object Relatives are represented as Folded Structures, Object Relatives are more costly than Subject Relatives -- even at the first level of embedding! Object Relatives require going out of the plane and coming back up as in the case of parallel strands! Optimization with respect to Folded Structures !!