Parsing Ill-formed Input without an Error Grammar or Mal-Rules:

1. Parsing Ill-formed Input without an Error Grammar or Mal-Rules: An Introduction to constraints, lexical grammar, chart parsing and a guy named Earley

2. How We Parsed Ill-formed Input thus far: • Pattern Matching (CLEF, Liou’s English Grammar Checker, CALLE, VINCI) • Finite State Automata (ARCTA) • Error Grammars / Mal-rules (German Tutor, FROG, FGA, etc. The above parsers are limited by their inflexibility; their inability to handle the unpredictable error.

3. So, how do we give a parser the elasticity it needs to perform [correct] error analysis? Constraint Relaxation We introduce the idea of: (BKSW, EPISTLE, etc.)

4. What is a Constraint? A constraint is “a condition which must be fulfilled in order for the parsing algorithm to take a certain action. This condition can be either simple or complex, that is, made up of several sub-conditions connected by logical operators.” (p. 66 Vandeventer Fatin)

5. Example taken from Vandeventer: a. [DP The cat] b. [Tbar [VP sleeps]] c. [TP [ DP The cat] [Tbar [VP sleeps]]] (1a) can combine with (1b) to form (1c) only if the complex Condition (2) is true: (2) (1a) must be a DP and (1b) must be a Tbar and (1a) and (1b) agree in number and (1a) and (1b) agree in person. --can be broken down further into sub-conditions

6. Types of Contraints: • Types of Constraints: • “Hard” Constraints- constraints when violated make for very ungrammatical sentences. ex.: *[TP[AdvP Very ] [Tbar [VP sleeps]]] • “Soft” Constraints: when violated, form “less grammatical” utterances- such as agreement- ex.: *[TP[DP The cat ] [Tbar [VP sleep]]] • “Conditional” Constraints: those which apply only when another prior condition is met- ex.: ? [TP[DP The desk ] [Tbar [VP sleeps]]]

7. So what’s “Constraint Relaxation, and what can it do for ICALL? Constraint Relaxation produces an environment where (relatively) “ungrammatical” sentences can be successfully parsed. The “hard”-ness of a violated constraint is what produces types of ungrammaticality- so identifying the constraint which is in violation allows for the parser to identify- simultaneously- the constraint violation and thus, the error. “Hard” constraints are usually not the ones relaxed. No new rules needed to be added to the grammar itself- since the parser now goes to “plan B” by itself.

8. Problems with constraint relaxation: Over-generation - increase in search space created by the relaxation itself. 3-element mismatch- verb/preposition/noun Not perfect- the parser still has to have a good knowledge of which constraints to relax, when, how often, etc. Not great from a cross-linguistic perspective; and still rather dependant upon some “pre-prescribed” strategies.

9. The Dawning of a new Grammar: LFG: Lexical Functional Grammar Lexical- so non-transformational- so alternations are encoded in the lexicon itself- not necessarily in the syntax Functional- very basic grammatical constructions- unlike phrase- structure rules Generally derives meaning from the combination of two main structures: C-structure= syntactic “constituent” structure- NPs, VPs and the like & F-structure= functional relations- feature-based structures

10. How it works: LFG allows for the ability to account for inflections and patterns without the use of transformations- this is encoded in the lexicon itself, and things like wh-words are both question and object words (What is that? That is what I was talking about yesterday.) F-structure: is governed by three key principles: (thanks, Markus) Completeness: all arguments share the same features Coherence: all arguments in a structure are licensed by the predicate Uniqueness: all attributes have a single value. F-structure and C-structure are mapped together.

11. The f-structure: the “pudding” of LFG: f-structure is function from attributes to values; we can use functional notation to constrain plausible f-structures to derive what is termed as a minimal solution- The value which satisfies the constraints of the f-structure and phrase structure introduced by the lexical items.

12. OK- so what’s an example? Let’s take “sighs”, and “Mary” We would posit the following lexical entry (constraints) for each: sighs: (f PRED) = ‘sigh<SUBJ>’ (f SUBJ CASE) = NOM (f SUBJ NUM) = SG Mary: (g PRED) = ‘Mary’ (g SUBJ NUM) = SG “Phrasal” constraints: (f SUBJ) = g We input our lexical values and run through our function…

13. (f SUBJ) = g (g SUBJ) = ‘MARY’ (g NUM) = SG (f PRED) = ‘SIGH<SUBJ>’ (g CASE) = NOM (g NUM) = SG And then came our minimal solution: PRED ‘SIGH<SUBJ>’ f: SUBJ g: PRED ‘MARY’ CASENOM NUMSG *Note: there is not always one solution which satisfies each f-description- but there is only one f-structure for every c-structure- but there can be multiple c-structures which satisfy an f-structure.

14. The mapping of the c-structure of the previous f-structure: PRED ‘SIGH<SUBJ>’ (IP): SUBJ [PRED ‘MARY’] (NP): SUBJ [PRED ‘MARY’]

15. OK- enough of that- So what does this mean for error recognition and parsing in CALL? The problems with the NLP side of CALL, for both Vandeventer Fatin and Reuer is that the parser has to determine whether flaw in the sentence which can’t be parsed is in the input or the parser itself- The goal would be to move away from grammars which rely heavily on error-anticipation, and whose analysis and terminology does not align with that of standard pedagogical categories- which require “translation” in itself for feedback.

16. So, why does Reuer like LFG? Particularly, what about LFG lends itself well to error analysis?

17. LFG has a few nice properties that make error recognition - and encoding much cleaner • Syntactic errors can be identifies in either the feature structure or the constituent structure. • Feature errors, or what he refers to as “Attribute Value Matrices” (AVMs) must undergo a unification process- and if it fails, then the error is recorded, an adjustment is made, and the parser can continue, with the error stored in its “memory”. • Unification allows for identifying the softer constraints of disagreement (person, number, etc) and also allows for the identification of “hard” constraints. • For c-structure, the parser can identify problems in ordering: insertion, omission, misplacement, etc.- and this is stored as an annotation in Earley’s chart parser. • LFG leaves nice, indestructible breadcrumbs- and the parser can easily find its way back to the student.

18. So what is Unification? • The idea is to keep tabs on all the instances when unification failed in the analysis- this activates the subsumption rule - if the feature is a mismatch, the erroneous one is injested and stored by the parser, the closest correct value is temporarily substituted to continue the parsing process (note: sometimes there is a subsumption when the 2 are equivalently, reflexively erroneous- and one is subsumed, and the parser continues, again). • For every error that is recorded, this information is also passed up, so that the parser may access higher syntactic nodes with knowledge it may encounter the error that was planted early; and that it can bleed throughout the tree, so it knows not record every impasse as a feature unification error. • Feature mismatch can be recognized between sister nodes- or with daughters with their parent nodes: ex from Reuer (p. 7): [subj: [pers: III]] pred: PRED(subj) = pred: PRED(subj) subj: [pers I] subj: pers: III err: [pers:I] _____________ err: [subj] • Errors in c-structure use the Earley SHIFT predicate chart-parsing algorithm

19. So LFG and CALL: Symbiotically Happy • LFG is rather good cross-linguistically- as most languages share common f-structure constraints- the difference is usually in the c-structure.(Reuer, p. 10) • LFG also uses terminology more aligned with stand language pedagogy-- the lexicon is the foundation upon which the system is built- not the place where exceptions are encoded. • Follows the example of syntactic phenomena tied directly to the behavior of lexemes- not how lexemes fit in the grammar, but how they define it.

20. Still, it’s not totally perfect… • Two problems still remain: • The insertion error is still a bit problematic (linear, c-structure)- have to find a way to scan, and then delete • Words with multiple functions (heavily inflected morphologically)-making the lexical list very long, and unification time-consuming; Reuer posits implementing “disjuntive rules”

21. Chart Parsing for Ill-Formed Input in ICALL Systems

22. Bidirectional Chart Parsing • Incremental Chart Parsing • Top-Down vs. Bottom-Up Chart Parsers (or both) • [Extended version of] Earley-Based Chart Parser • For the processing of ps-rules • Clearly, recognition of linearization errors (word order) is an important role in error feedback with ICALL systems. • But we also get to ‘learn’ more errors during each parsing process. • The type of error can be stored in the chart when a parse fails. • [Variant of the] Viterbi Algorithm • Cocke-Kasami-Younger (CKY) Algorithm

23. Advantages of Chart Parsing • It avoids multiplication of parsing effort • It provides a compact representation for ‘local ambiguity’ • It provides a representation for ‘partial parses’

24. 1. No Redundancy Chart Parsing is a forward-moving process that does not re-analyze any parse that has been added to the chart as either a successful parse or mistake. • Each possible parse is analyzed, however, it is analyzed once. Other methods don’t keep track of what has already been seen. Therefore, they may • “Rediscover” phrases already successfully parsed, and • Re-explore hypotheses already failed.

25. 2. Inadequacy of trees as representations of local ambiguities • Multiple parses need separate trees for each possibility for constituent’s parse. • ** Not possible to collapse trees to share common information For Example: “duck under the chair” has two possible trees since the PP is not a separate constituent. Possibility 1: Possibility 2: NP N PP duck P NP under D N the chair VP PP V P NP duck under D N the chair

26. 3. Inadequacy of trees as representations when there are incomplete structures • Incomplete structures result when the input cannot be parsed either because of ill-formed input or an insufficient grammar. • Trees cannot ‘flag’ where these errors occur, the parse will fail and there is no memory of violation for, for example, error feedback.

27. General Idea behind Chart Parsing… • Scan each terminal node (lexical item) input into the parser. • Predict the rule/constituent • How it fits in with the completed input • Complete the addition of this item as it fits into the grammar and what has already been seen. See example at: http://nltk.sourceforge.net/screenshots/chart.jpg

29. A Virtual Memory • Always forward moving (Dot Rules) • Anything that is completed is stored in a chart. • Chart Parsers can capture information through this that makes its own “mal-rules” as it goes. • If a parse is not successful, you still gain any information already stored in the chart by the parser.

30. Disadvantages in Chart Parsing Since everything is always “forward-moving”, we take care of redundancy. However, everything that could be done is done, much of which is useless. • Note, since there is no repetition, everything that can be done is only done once.

31. So…What does this all mean to ICALL Systems? • Linearization Errors (c-structure) • Chart Parsing in ICALL systems is used strictly for syntactic purposes • Word-Order is encoded in the chart, not feature structures • Error Feedback • CALL systems are not able to provide the learner with individualized teaching and flexible feedback, but the use of Chart Parsing in ICALL systems could achieve this. • Using processes such as Chart Parsing, ICALL systems can pin-point syntactic errors, keep track of ill-formed input (individually or language-based) and provide personalized feedback for a more effective system. • Example ICALL System: ESPADA

32. ESPADA • Knowledge-based intelligent error feedback in a Spanish ICALL system [Koller, 2003] • A Spanish ICALL system which helps improvement of syntactical knowledge. • Aim: Development and implementation of a fully functional syntax teaching system for German learners of Spanish. • Focuses on adult learners with a basic linguistic knowledge of grammar and syntactic structures. • Curriculum-independent resource for learning and practicing syntactical structures. • Consists of three components: • Demonstration Module: Provides animated presentation of selected grammatical information. • Lexical Module: Stores and selectively displays lexical information. • Analysis Module: Analyzes the user’s input and generates appropriate feedback.

33. ESPADA: The Analysis Module (1) • The Analysis Module analyzes the user’s input and returns feedback on ill-formed sentences. • Parses the input using a chart-based island parser with mal-rules and constraint relaxation. • Can recognize and generate feedback on 28 different error types from different linguistic levels. • Analysis focused on simple Spanish sentences.

34. ESPADA: The Analysis Module (2) Steps involved in the analysis process… • Word Forms: Syntactic and semantic information of input’s words forms. • Variants: Checks for POS ambiguities, asks user to choose intended POS if multiple are possible for a word. • POS Filtering: Positions of negation adverbs and/or adverbial complements are saved (for reference) then the words are removed. All remaining words in the input are then part of the subject, verbal group, or an object. • Verbal Group: Checked through various if-then-tests to make sure this group meets the requirements of expected grammar. • Negation Adverbs and Object Pronouns: Number and position checked simultaneously (positions should depend on each other) • Determining Sentence Patterns: If no errors were found up to this point, the basic sentence pattern is extracted and this pattern is translated into phrase patterns (i.e., SVD translates to NP|NP and into NP|PP, where ‘|’ symbolizes the verbal group). • Parsing: Input is parsed as two different groups: pre-verbal and post-verbal. [more details to follow] • Tests: If parsing is successful, tests are executed to detect errors in agreement, syntax or semantics. • Displaying Results: The sentence pattern with the minimal errors is displayed.

35. ESPADA: The Parser • Chart-based island parser [Earley Algorithm] • Analysis of non-verbal groups of the input. • The parsing process is run twice. Pre-Verbal Part: • parsed against those parts of al phrase patterns which are on the left-hand-side of the Verbal Group (separated by “|” in the rules) Post-Verbal Part: • Parsed against those phrase patterns which were parsed successfully in the first parsing phrase. • Matched against the part of every phrase pattern which are on the right-hand-side of the Verbal Group. Example: “El padre dio un libro a su hijo” and the pattern NP|NP;PP. • First, El padre is parsed against NP. • If this parse is successful, un libro a su hijo is parsed against NP;PP

36. ESPADA: Mal-Rules • The mal-rules recognize faulty structures of sentence components and nominal phrases. • Mal-rules for Nominal Phrases • Can recognize errors in NPs having to do with either word order or missing words. • New mal-rules can be integrated to adapt the grammar so L1-specific errors are recognized or errors in differing NP structures in other languages • Mal-rules for Sentence Components • Used to parse sentences where the type and/or number of recognized complements does not correspond to the required type and number in the sentence patterns of the main verb. • Finite amount of possible sentence structures for simple sentences [set of all ‘meaningful’ combinations of the set of complements comprising Subject, Direct Object, Indirect Object, Prepositional Object, Necessary Adverbial Complement. • After extracting the regular sentence patterns of the input, all other possible sentence patterns are added to the chart explicitly labelled as mal-rules. • If any regular sentence pattern has been parsed successfully, only these patterns are then checked thoroughly for syntactic, semantic, and agreement errors

37. ESPADA: Constraint Relaxation • Constrain Relaxation is implemented in terms of successive, layered constrain application and constraint violation bookkeeping. • No constraints are applied at all during initial parsing, only after parsing has been completed (step 3 of chart parsing steps). • After parsing is completed, syntactic and semantic adequacy and agreement requirements are controlled through multiple tests. • After these tests have been done, the constraint violations are counted for the succcessfully parsed sentence pattern. • The sentence pattern(s) with the lowest number of constraint violations are then displayed

38. ESPADA: Generating Feedback Four different types of feedback are generated after parsing and all constraint checks have been done: • The sentence pattern could be parsed with at least one regular sentence pattern and no constraint violations were detected. • Feedback is the successfully parsed sentence and the corresponding sentence components. • The sentence could be parsed successfully with at least one regular sentence pattern, but constraint violations were detected. • Feedback is the sentence with lowest number of constraint violations with the errors displayed with the cascaded feedback system*. • The sentence could only be parsed successfully with a mal-rule on sentence level. • Feedback is the recognized phrases and offers further general information. • It was not possible to parse the sentence with any sentence patter (either regular or mal-rule based). • Feedback is about the failure of analysis and offers general information about the possible reasons for failure.

39. Figure 5. Feedback from Analysis Module Results of Analysis Module Analysis Module Your sentence could be analyzed. It just has some mistakes. Sentence Pattern Sentence Complements subject el padres verb hablamos prepositional object con su hijo Mistake: agreement within sentence component Mistake: agreement subject-verb SVP JavaScript Alert x In the subject el padres padres and el differ in number. padres: plural el: singular ! OK