1 / 20

Lecture 2

Lecture 2. Regular Expressions and Automata in Language Analysis. Statistical vs. Symbolic (Knowledge Rich) Techniques. How much linguistic knowledge do our representations and algorithms need to have to do ‘successful’ NLP? Bill hit John. John, Bill hit.

zofia
Download Presentation

Lecture 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 2 Regular Expressions and Automata in Language Analysis CS 4705

  2. Statistical vs. Symbolic (Knowledge Rich) Techniques • How much linguistic knowledge do our representations and algorithms need to have to do ‘successful’ NLP? • Bill hit John. • John, Bill hit. • 80/20 Rule: when do we need to worry about the other 20%?

  3. Today • Review some of the simple representations and ask ourselves how we might use them to do interesting and useful things • Regular Expressions • Finite State Automata • Think about the limits of these simple approaches: when do we need more?

  4. Uses of Regular Expressions in NLP • Simple but powerful tools for large corpus analysis -- ‘shallow’ processing • What word is most likely to begin a sentence? • What word is most likely to begin a question? • How often do people end sentences with prepositions? • With other simple statistical tools, allow us to • Obtain word frequency and co-occurrence statistics • Build simple interactive applications (e.g. Eliza) • Authorship: Who wrote Shakespeare’s plays? The Federalist papers? The Unibomber letters?

  5. RE Matches Possible use /./ Any character A non-blank line /\./, /\?/ A ‘.’, a ‘?’ /[bckmsr]/ Any of these chars Rhyme:/[bckmrs]ite/ /[a-z]/ Any l.c. letter /[A-Z]/ Any u.c. letter /[A-Z][a-z]*/ /[^A-Z]/ Any non-u.c. char /[^A-Z][a-z]*/ Review A statement, a question Rhyme: /[a-z]ite/

  6. RE plus E.G. /kitt(y|ies)/ Morphological variants of ‘kitty’ -- but / (.+ier) and \1 / Patterns: happier and happier, fuzzier and fuzzier, classifier and classifier

  7. Substitutions (Transductions) • E.g. unix sed or ‘s’ operator in Perl • s/regexp1/pattern/ • s/I am feeling (.+)/You are feeling \1 ?/ • s/I gave (.+) to (.+)/Why would you give \2 \1 ?/ • s/You are (.+)[.]*/Why would you say that I am \1?/ • s/([1]?[0-9]) o’clock ([AaPp][. ]*[Mm][. ]*)/\1:00 \2/ • How would you convert to 24-hour clock? • s/[0-9][0-9][0-9]-[0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]/ <guess>

  8. Examples • Predictions from a news corpus: • Which candidate for President is mentioned most often in the news? Is going to win? • What stock should you buy? • Which White House advisers have the most power? • Language use: • Which form of comparative is more frequent: ‘Xer’ or ‘moreX’? • Which pronouns occur most often in subject position? • How often do sentences end with infinitival ‘to’? • What words most often begin and end sentences? • What are the 20 most common words in your email? In the news? In Shakespeare’s plays?

  9. Emotional language: • What words indicate what emotions? • Happiness • Anger • Confidence • Despair • How can we identify emotions automatically?

  10. a b a a ! q0 q1 q2 q3 q4 Finite State Automata • FSAs recognize the regular languages represented by regular expressions • SheepTalk: /baa+!/ • Directed graph with labeled nodes and arc transitions • Five states: q0 the start state, q4 the final state, 5 transitions

  11. a b a a ! q0 q1 q2 q3 q4 Formally • FSA is a 5-tuple consisting of • Q: set of states {q0,q1,q2,q3,q4} • : an alphabet of symbols {a,b,!} • q0: a start state in Q • F: a set of final states in Q {q4} • (q,i): a transition function mapping Q x  to Q

  12. FSA recognizes (accepts) strings of a regular language • baa! • baaa! • baaaa! • … • Tape metaphor: will this input be accepted?

  13. State Transition Table for SheepTalk

  14. a b a a ! q0 q1 q2 q3 q4 Non-Deterministic FSAs for SheepTalk b a a ! q0 q1 q2 q3 q4 

  15. Problems of Non-Determinism • At any choice point, we may follow the wrong arc • Potential solutions: • Save backup states at each choice point • Look-aheadin the input before making choice • Pursue alternatives in parallel • Determinizeour NFSAs (and thenminimize) • FSAs can be useful tools for recognizing – and generating – subsets of natural language • But they cannot represent all NL phenomena (center embedding: The mouse the cat chased died.)

  16. Simple vs. linguistically rich representations…. • How do we decide what we need?

  17. FSAs as Grammars for Natural Language dr the rev mr pat l. robinson q0 q1 q2 q3 q4 q5 q6 ms hon  mrs 

  18. If we want to extract all the proper names in the news, will this work? • What will it miss? • Will it accept something that is not a proper name? • How would you change it to accept all proper names without false positives? • Precision vs. recall….

  19. Summing Up • Regular expressions and FSAs can represent subsets of natural language as well as regular languages • Both representations may be impossible for humans to understand for any real subset of a language • But they are relatively easy to use for small subsets • Can be hard to scale up: when many choices at any point (e.g. surnames) • Next time: Read Ch 3

More Related