1 / 91

CS60057 Speech &Natural Language Processing

CS60057 Speech &Natural Language Processing. Autumn 2007. Lecture 16 5 September 2007. Parsing with features. We need to constrain the rules in CFGs, for example to coerce agreement within and between constituents to pass features around to enforce subcategorisation constraints

mariko
Download Presentation

CS60057 Speech &Natural Language Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS60057Speech &Natural Language Processing Autumn 2007 Lecture 16 5 September 2007 Natural Language Processing

  2. Parsing with features • We need to constrain the rules in CFGs, for example • to coerce agreement within and between constituents • to pass features around • to enforce subcategorisation constraints • Features can be easily added to our grammars • And later we’ll see that feature bundles can completely replace constituents Natural Language Processing

  3. Parsing with features • Rules can stipulate values, or placeholders (variables) for values • Features can be used within the rule, or passed up via the mother nodes • Example: subject-verb agreement S  NP VP [if NP and VP agree in number] number of NP depends on noun and/or determiner number of VP depends on verb S  NP(num=X) VP (num=X) NP (num=X)  det(num=X) n (num=X) VP(num=X)  v(num=X) NP(num=?) Natural Language Processing

  4. this man n(num=pl) men Declarative nature of features NP (num=X)  det(num=X) n (num=X) The rules can be used in various ways • To build an NP only if det and n agree (bottom-up) • When generating an NP, to choose an n which agrees with the det (if working L-to-R) (top-down) • To show that the num value for an NP comes from its components (percolation) • To ensure that the num value is correctly set when generating an NP (inheritance) • To block ill-formed input this det (num=sg) these det (num=pl) the det (num=?) man n (num=sg) men n (num=pl) NP (num=sg) det(num=sg) n(num=sg) Natural Language Processing

  5. the man Use of variables NP (num=X)  det(num=X) n (num=X) • Unbound (unassigned) variables (ie variables with a free value): • the can combine with any value for num • Unification means that the num value for the is set to sg this det (num=sg) these det (num=pl) the det (num=?) man n (num=sg) men n (num=pl) NP (num=sg) det(num=?) n(num=sg) Natural Language Processing

  6. Parsing with features • Features must be compatible • Formalism should allow features to remain unspecified • Feature mismatch can be used to block false analyses, and disambiguate • e.g. they can fish ~ he can fish ~ he cans fish • Formalism may have attribute-value pairs, or rely on argument position e.g. NP(_num,_sem)  det(_num) n (_num,_sem) an = det(sing) the = det(_num) man = n(sing,hum) Natural Language Processing

  7. Parsing with features VP  v e.g. dance VP v NP e.g. eat VP  v NP NP e.g. give VP  v PP e.g. wait (for) • Using features to impose subcategorization constraints VP(_num)  v(_num,intr) VP(_num)  v(_num,trans) NP VP(_num)  v(_num,ditrans) NP NP VP(_num)  v(_num,prepobj(_case)) PP(_case) PP(_case)  prep(_case) NP dance = v(plur,intr) dances = v(sing,intr) danced = v(_num,intr) waits = v(sing,prepobj(for)) for = prep(for) Natural Language Processing

  8. S NP VP (_num) (_num) v (sing,intrans) v NP (sing,trans) (_1) det n (_num) (_num) det n (_1) (_1) shot the man those elephants Parsing with features (top-down) S  NP(_num) VP(_num) NP(_num)  det(_num) n(_num) VP(_num)  v(_num,intrans) VP(_num)  v (_num,trans) NP(_1) the man shot those elephants S  NP(_num) VP(_num) NP(_num)  det(_num) n(_num) the = det(_num) man = n(sing) (sing) (sing) _num=sing VP(sing)  v(sing,intrans) shot = v(sing,trans) (sing) (pl) VP(sing)  v(sing,trans) NP(_1) (sing) shot = v(sing,trans) NP(_1)  det(_1) n(_1) (pl) (pl) those = det(pl) elephants = n(pl) Natural Language Processing

  9. ATTR1 VAL1 ATTR2 VAL2 ATTR3 VAL3 NUM SG PERS 3 CAT NP NUMBER SG PERSON 3 CAT NP AGR Feature structures • Instead of attaching features to the symbols, we can parse with symbols made up entirely of attribute-value pairs: “feature structures” • Can be used in the same way as seen previously • Values can be atomic … • … or embedded feature structures … Natural Language Processing

  10. UnificationProbabilistic CFG August 31, 2006 Natural Language Processing

  11. Feature Structures • A set of feature-value pairs • No feature occurs in more than one feature-value pair (a partial function from features to values) • Circular structures are prohibited. Natural Language Processing

  12. Structured Feature Structure Part of a third-person singular NP: Natural Language Processing

  13. Reentrant Feature Structure • Two features can share a feature structure as value. Not the same thing as them having equivalent values! • Two distinct feature structure values: • One shared value (reentrant feature structure): Natural Language Processing

  14. AGR 1 SUBJ [ AGR 1 ] NUM SG PERS 3 • they can be coindexed CAT S HEAD Natural Language Processing

  15. AGR 1 SUBJ [ AGR 1 ] NUM SG PERS 3 Parsing with feature structures • Grammar rules can specify assignments to or equations between feature structures • Expressed as “feature paths” e.g. HEAD.AGR.NUM = SG CAT S HEAD Natural Language Processing

  16. Subsumption () • (Partial) ordering of feature structures • Based on relative specificity • The second structure carry less information, but is more general (or subsumes) the first. Natural Language Processing

  17. Subsumption • A more abstract (less specific) feature structure subsumes an equally or more specific one. • A feature structure F subsumes a feature structure G ( F  G) if and only if : • For every structure x in F, F(x) G(x) (where F(x) means the value of the feature x of the feature structure F). • For all paths p and q in F such that F(p)=F(q), it is also the case that G(p)=G(q). • An atomic feature structure neither subsumes nor is subsumed by another atomic feature structure. • Variables subsume all other feature structures. • A feature structure F subsumes a feature structure G (F  G) if if all parts of F subsumes all parts of G. Natural Language Processing

  18. Subsumption Example Consider the following feature structures: (1) (2) (3) (1)  (3) (2)  (3) but there is no subsumption relation between (1) and (2) Natural Language Processing

  19. Feature Structures in The Grammar • We will incorporate the feature structures and the unification process as follows: • All constituents (non-terminals) will be associated with feature structures. • Sets of unification constraints will be associated with grammar rules, and these rules must be satisfied for the rule to be satisfied. • These attachments accomplish the following goals: • To associate feature structures with both lexical items and instances of grammatical categories. • To guide the composition of feature structures for larger grammatical constituents based on the feature structures of their component parts. • To enforce compatibility constraints between specified parts of grammatical constraints. Natural Language Processing

  20. CAT NP NUMBER ?? PERSON 3 NUMBER SG PERSON 3 CAT NP NUMBER SG PERSON 3 Feature unification • Feature structures can be unified if • They have like-named attributes that have the same value: [NUM SG]  [NUM SG] = [NUM SG] • Like-named attributes that are “open” get the value assigned:  = Natural Language Processing

  21. NUM SG PERS 3 CAT NP NUMBER SG CAT NP NUMBER SG PERSON 3 CAT NP AGR Feature unification • Complementary features are brought together • Unification is recursive • Coindexed structures are identical (not just copies): assignment to one effects all  = [PERSON 3] CAT NP AGR [NUM SG] CAT NP AGR [PERS 3]  = Natural Language Processing

  22. VAL INDEF NUM SG Example CAT N AGR _2 SEM _3 CAT NP AGR _1  _2 SEM _3 CAT DET AGR _1  CAT DET AGR  a CAT DET AGR [VAL DEF]  the CAT N LEX “man” AGR [NUM SG] SEM HUM  man Natural Language Processing

  23. CAT DET AGR [VAL DEF]  the CAT N LEX “man” AGR [NUM SG] SEM HUM  man the man the the man CAT N AGR _2 SEM _3 CAT NP AGR _1  _2 SEM [_3] CAT DET AGR _1  LEX “man” AGR [NUM SG] SEM HUM VAL DEF [VAL DEF] NUM SG HUM Natural Language Processing

  24. CAT DET AGR  a VAL INDEF AGR NUM SG VAL INDEF NUM SG VAL INDEF NUM SG VAL INDEF NUM SG CAT N LEX “man” AGR [NUM SG] SEM HUM  man a a man a man CAT NP AGR _1  _2 SEM [_3] CAT N AGR _2 SEM _3 CAT DET AGR _1  LEX “man” AGR [NUM SG] SEM HUM [NUM SG] HUM Natural Language Processing

  25. Types and inheritance • Feature typing allows us to constrain possible values a feature can have • e.g. num = {sing,plur} • Allows grammars to be checked for consistency, and can make parsing easier • We can express general “feature co-occurrence conditions” … • And “feature inheritance rules” • Both these allow us to make the grammar more compact Natural Language Processing

  26. Co-occurrence conditions and Inheritance rules • General rules (beyond simple unification) which apply automatically, and so do not need to be stated (and repeated) in each rule or lexical entry • Examples: [cat=np]  [num=??, gen=??, case=??] [cat=v,num=sg]  [tns=pres] [attr1=val1]   [attr2=val2] Natural Language Processing

  27. Inheritance rules • Inheritance rules can be over-ridden e.g. [cat=n]  [gen=??,sem=??] sex={male,female} gen={masc,fem,neut} [cat=n,gen=fem,sem=hum]  [sex=female] uxor [cat=n,gen=fem,sem=hum] agricola [cat=n,gen=fem,sem=hum,sex=male] Natural Language Processing

  28. Unification in Linguistics • Lexical Functional Grammar • If interested, see PARGRAM project • GPSG, HPSG • Construction Grammar • Categorial Grammar Natural Language Processing

  29. Unification • Joining the contents of two feature structures into one new (the union of the two originals). • The union is most general feature structure subsumed by both. • The union of two contradictory feature structures is undefined (unification fails). Natural Language Processing

  30. Unification Constraints • Each grammar rule will be associated with a set of unification constraints. 0  1 … n {set of unification constraints} • Each unification constraint will be in one of the following forms. < i feature path> = Atomic value < i feature path> = < j feature path> Natural Language Processing

  31. Unification Constraints -- Example • For example, the following rule S  NP VP Only if the number of the NP is equal to the number of the VP. will be represented as follows: S  NP VP <NP NUMBER> = <VP NUMBER> Natural Language Processing

  32. Agreement Constraints S  NP VP <NP NUMBER> = <VP NUMBER> S  Aux NP VP <Aux AGREEMENT> = <NP AGREEMENT> NP  Det NOMINAL <Det AGREEMENT> = <NOMINAL AGREEMENT> <NP AGREEMENT> = <NOMINAL AGREEMENT> NOMINAL  Noun <NOMINAL AGREEMENT> = <Noun AGREEMENT> VP  Verb NP <VP AGREEMENT> = <Verb AGREEMENT> Natural Language Processing

  33. Agreement Constraints -- Lexicon Entries Aux  does <Aux AGREEMENT NUMBER> = SG <Aux AGREEMENT PERSON> = 3 Aux  do <Aux AGREEMENT NUMBER> = PL Det  these <Det AGREEMENT NUMBER> = PL Det  this <Det AGREEMENT NUMBER> = SG Verb  serves <Verb AGREEMENT NUMBER> = SG <Verb AGREEMENT PERSON> = 3 Verb  serve <Verb AGREEMENT NUMBER> = PL Noun  flights <Noun AGREEMENT NUMBER> = PL Noun  flight <Noun AGREEMENT NUMBER> = SG Natural Language Processing

  34. Head Features • Certain features are copied from children to parent in feature structures. Example: • AGREEMENT feature in NOMINAL is copied into NP. • The features for most grammatical categories are copied from one of the children to the parent. • The child that provides the features is called head of the phrase, and the features copied are referred to as head features. • A verb is a head of a verb phrase, and a nominal is a head of a noun phrase. We may reflect these constructs in feature structures as follows: NP  Det NOMINAL <Det HEAD AGREEMENT> = <NOMINAL HEAD AGREEMENT> <NP HEAD> = <NOMINAL HEAD> VP  Verb NP <VP HEAD> = <Verb HEAD> Natural Language Processing

  35. SubCategorization Constraints • For verb phrases, we can represent subcategorization constraints using three techniques: • Atomic Subcat Symbols • Encoding Subcat lists as feature structures • Minimal Rule Approach (using lists directly) • We may use any of these representations. Natural Language Processing

  36. Atomic Subcat Symbols VP  Verb <VP HEAD> = <Verb HEAD> <VP HEAD SUBCAT> = INTRANS VP  Verb NP <VP HEAD> = <Verb HEAD> <VP HEAD SUBCAT> = TRANS VP  Verb NP NP <VP HEAD> = <Verb HEAD> <VP HEAD SUBCAT> = DITRANS Verb  slept <Verb HEAD SUBCAT> = INTRANS Verb  served <Verb HEAD SUBCAT> = TRANS Verb  gave <Verb HEAD SUBCAT> = DITRANS Natural Language Processing

  37. Encoding Subcat Lists as Features Verb  gave <Verb HEAD SUBCAT FIRST CAT> = NP <Verb HEAD SUBCAT SECOND CAT> = NP <Verb HEAD SUBCAT THIRD> = END VP  Verb NP NP <VP HEAD> = <Verb HEAD> <VP HEAD SUBCAT FIRST CAT> = <NP CAT> <VP HEAD SUBCAT SECOND CAT> = <NP CAT> <VP HEAD SUBCAT THIRD> = END • We are only encoding lists using positional features Natural Language Processing

  38. Minimal Rule Approach • In fact, we do not use symbols like SECOND, THIRD. They are just used to encode lists. We can use lists directly (similar to LISP). <SUBCAT FIRST CAT> = NP <SUBCAT REST FIRST CAT> = NP <SUBCAT REST REST> = END Natural Language Processing

  39. Subcategorization Frames for Lexical Entries • We can use two different notations to represent subcategorization frames for lexical entries (verbs). Verb  want <Verb HEAD SUBCAT FIRST CAT> = NP Verb  want <Verb HEAD SUBCAT FIRST CAT> = VP <Verb HEAD SUBCAT FIRST FORM> = INFINITITIVE Natural Language Processing

  40. Implementing Unification • The representation we have used cannot facilitate the destructive merger aspect of unification algorithm. • For this reason, we add additional features (additional edges to DAGs) into our feature structures. • Each feature structure will consists of two fields: • Content Field -- This field can be NULL or may contain ordinary feature structure. • Pointer Field -- This field can be NULL or may contain a pointer into another feature structure. • If the pointer field of a DAG is NULL, the content field of DAG contains the actual feature structure to be processed. • If the pointer field of a DAG is not NULL, the destination of that pointer represents the actual feature structure to be processed. Natural Language Processing

  41. Extended Feature Structures  Natural Language Processing

  42. SG C Num    P C Null    3 Per C P    Null P Null Extended DAG Natural Language Processing

  43. 3 SG C C     Num     P P C Null Null     Null Per P C P Null Unification of Extended DAGs Natural Language Processing

  44. 3 SG Per C C  Null     C  Num     P P C Null Null P  Null     P Null Per P C P Null Unification of Extended DAGs (cont.) Natural Language Processing

  45. Unification Algorithm functionUNIFY(f1,f2)returns fstructure or failure f1real real contents of f1 /* dereference f1 */ f2real real contents of f2 /* dereference f2 */ if f1real is Null then { f1.pointer  f2; returnf2; } else iff2real is Null then { f2.pointer  f1; returnf1; } else if f1real and f2real are identical then { f1.pointer  f2; returnf2; } else if f1real and f2real are complex feature structures then { f2.pointer  f1; for eachfeaturein f2realdo { otherfeature  Find or create a feature corresponding to feature in f1real; if UNIFY(feature.value,otherfeature.value) returns failure then return failure; } return f1; } elsereturn failure; Natural Language Processing

  46. Example - Unification of Complex Structures Natural Language Processing

  47. C C Agr C Sub C Num C Agr Agr C C C C Per • • • • • • • • • • • • • • • • SG • Null C Per • • • • • • • • • • Sub Null Null Null Null Null Null Null Null Null • Null • 3 Example - Unification of Complex Structures (cont.) Natural Language Processing

  48. Parsing with Unification Constraints • Let us assume that we have augmented our grammar with sets of unification constraints. • What changes do we need to make a parser to make use of them? • Building feature structures and associate them with sub-trees. • Unifying feature structures when sub-trees are created. • Blocking ill-formed constituents Natural Language Processing

  49. Earley Parsing with Unification Constraints • What do we have to do to integrate unification constraints with Early Parser? • Building feature structures (represented as DAGs) and associate them with states in the chart. • Unifying feature structures as states are advanced in the chart. • Blocking ill-formed states from entering the chart. • The main change will be in COMPLETER function of Earley Parser. This routine will invoke the unifier to unify two feature structures. Natural Language Processing

  50. Building Feature Structures NP  Det NOMINAL <Det HEAD AGREEMENT> = <NOMINAL HEAD AGREEMENT> <NP HEAD> = <NOMINAL HEAD> corresponds to Natural Language Processing

More Related