1 / 128

Information Extraction

Information Extraction. 2 sessions in the section “Web Search” of the course “Web Mining” at the École nationale supérieure des Télécommunications in Paris/France in fall 2010 by Fabian M. Suchanek. This document is available under a

marcin
Download Presentation

Information Extraction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Information Extraction 2 sessions in the section “Web Search” of the course “Web Mining” at the Écolenationalesupérieure des Télécommunications in Paris/France in fall 2010 by Fabian M. Suchanek This document is available under a Creative Commons Attribution Non-Commercial License

  2. Organisation • 4h class on Information extraction 2 sessions with 2h each • Small home-work given at the end of each session, to be handed in for the next session (on paper or by email) • Web-site: http://suchanek.name/ Teaching

  3. Motivation Elvis, when I need you, I can hear you! Elvis Presley 1935 - 1977 Will there ever be someone like him again?

  4. Motivation Another Elvis Elvis Presley: The Early Years Elvis spent more weeks at the top of the charts than any other artist. www.fiftiesweb.com/elvis.htm

  5. Motivation Another singer called Elvis, young Personal relationships of Elvis Presley – Wikipedia ...when Elvis was a young teen.... another girl whom the singer's mother hoped Presley would .... The writer called Elvis "a hillbilly cat”en.wikipedia.org/.../Personal_relationships_of_Elvis_Presley

  6. Motivation SELECT * FROM person WHERE gName=‘Elvis’ AND occupation=‘singer’ Another Elvis Information Extraction 1: Elvis Presley 2: Elvis ... 3. Elvis ... ✗

  7. Motivation: Definition Information Extraction (IE) is the process of extracting structured information (e.g., database tables) from unstructured machine-readable documents (e.g., Web documents). Elvis Presley was a famous rock singer. ... Mary once remarked that the only attractive thing about the painter Elvis Hunter was his first name. Information Extraction

  8. Motivation: Examples

  9. Motivation: Examples

  10. Motivation: Examples

  11. Motivation: Examples

  12. Information Extraction and beyond Ontological Information Extraction Information Extraction (IE) is the process of extracting structured information (e.g., database tables) from unstructured machine-readable documents (e.g., Web documents). Fact Extraction Instance Extraction Named Entity Recognition Tokenization& Normalization ...married Elvis on 1967-05-01 Source Selection 05/01/67  1967-05-01 ?

  13. Sources: The Web (1 trillion Web sites) Source for the languages: http://www.clickz.com/clickz/stats/1697080/web-pages-language Need not be correct

  14. Sources: Language detection Elvis Presley isteinerdergrößtenRockstarsallerZeiten. a bcäöüß ... How can we find out the language of a document? • Watch for certain characters or scripts (umlauts, Chinese characters etc.) • But: These are not always specific • Use the meta-information associated with a Web page • But: This is usually not very reliable • Use a dictionary • But: This is costly • Use frequent character signatures • (Count how often each character appears in the document. • Compare this histogram to the histogram computed on a • large text document corpus of the language in question) • Extension: Make a histogram of character n-grams • (n-gram: a sequence of n characters)

  15. Sources: Scripts Elvis Presley was a rock star. 猫王是摇滚明星 אלביסהיהכוכברוק وكانألفيسبريسلينجمالروك 록 스타 엘비스 프레슬리 ( Elvis Presley ถูกดาวร็อก (Latin script) (Chinese script, “simplified”) (Hebrew) (Arabic) (Korean script) (Thai script) Source: http://translate.bing.com Probably not correct

  16. Sources: Character Encodings ? 100,000 different characters from 90 scripts One byte with 8 bits per character (can store numbers 0-255) • How can we encode so many characters in 8 bits? • Ignore all non-English characters • There are 26 letters, + 26 lowercase letters + punctuation ≈ 100 chars • ... 65=A, 66=B, 67=C, ... ASCII standard ✓ • Depending on the script (the so-called code page), (Example) • the numbers mean different characters • Latin code page: ...., 65=A, 66=B, ... • Greek code page: ...., 65=α, ... Code page model ✓ • Invent special names for special characters • è = èHTML entity encoding✓

  17. Sources: Character Encodings ? 100,000 different characters from 90 scripts One byte with 8 bits per character (can store numbers 0-255) How can we encode so many characters in 8 bits? • Use 4 bytes to represent a character (Example) • ...65=A, 66=B, ..., 1001=α, ..., 2001=리 Unicode standard ✓ • then compress them into 1-4 bytes • ...65=A, 66=B, ..., 00+01=α, ..., 01+01=리 UTF-8 standard ✓ • or refer to the characters by their number • ϩ = αHTML entity encoding (too) ✓

  18. Sources: UTF-8 • Characters 0-0x7F, 7 bits: Latin alphabet, punctuation and numbers • 0xxxxxxx (i.e., equal to ASCII and most code pages) • Characters 0x80-0x7FF, 11 bits: Greek, Arabic, Hebrew, etc. • 110xxxxx 10xxxxxx (i.e., marker byte + follower byte) • Characters 0x800-0xFFFF, 16 bits: Chinese, Chinese and Chinese (et al) • 1110xxxx 10xxxxxx 10xxxxxx (i.e., marker byte + 2 follower bytes) • Advantages: • common Western characters require only 1 byte () • backwards compatibility with ASCII • stream readability (follower bytes cannot be confused with marker bytes) • sorting compliance We will assume that the document is a sequence of characters

  19. Sources: Structured Name Number D. Johnson 30714 J. Smith 20934 S. Shenker 20259 Y. Wang 19471 J. Lee 18969 A. Gupta 18884 R. Rivest 18038 H. Zhang 17902 L. Zhang 17800 J. Ullman 16804 Information Extraction TSV file (tab separated values) Related: CSV (comma separated values)

  20. Sources: Semi-Structured <catalog> <cd> <title> Empire Burlesque </title> <artist> <firstName> Bob </firstName> <lastName> Dylan </lastName> <artist> </cd> ... Information Extraction XML file (Extensible Markup Language) Related: YAML (YamlAin’t a Markup Language)

  21. Sources: Semi-Structured <table> <tr> <td> 2008-11-24 <td> Miles away <td> 7 <tr> ... Information Extraction HTML file with table (Extensible Markup Language) Wiki file with table (A Markup Language used in Wikipedia)

  22. Sources: “Unstructured” Founded in 1215 as a colony of Genoa, Monaco has been ruled by the House of Grimaldi since 1297, except when under French control from 1789 to 1814. Designated as a protectorate of Sardinia from 1815 until 1860 by the Treaty of Vienna, Monaco's sovereignty was recognized by the Franco-Monegasque Treaty of 1861. The Prince of Monaco was an absolute ruler until a constitution was promulgated in 1911. Information Extraction HTML file or text file or word processing document

  23. Sources: Mixed <table> <tr> <td> Professor. Computational Neuroscience, ... ... Information Extraction HTML file or text file or word processing document Different IE approaches work with different types of sources

  24. Sources: Domain Restricted to one Internet Domain (e.g., Amazon.com) Restricted to one Language (e.g., English) Restricted to one Thematic Domain (e.g., biographies) (Slide taken from William Cohen)

  25. Sources: Finding the Sources Information Extraction ? How can we find the documents to extract information from? • The document collection can be given a priori • (Closed Information Extraction) • e.g., a given Internet domain, all files on my computer, ... • We can aim to extract information from the entire Web • (Open Information Extraction) • The system can find by itself the source documents • e.g., by using an Internet search engine such as Google

  26. Information Extraction and beyond Ontological Information Extraction Information Extraction (IE) is the process of extracting structured information (e.g., database tables) from unstructured machine-readable documents (e.g., Web documents). Fact Extraction Instance Extraction Named Entity Recognition ✓ Tokenization& Normalization ...married Elvis on 1967-05-01 Source Selection 05/01/67  1967-05-01 ?

  27. Tokenization Tokenization is the process of splitting a text into tokens (i.e., words, punctuation symbols, identifiers and literals). On 2010-01-01 , President Sarkozy spoke this example sentence . • Challenges: • In some languages (Chinese, Japanese), • words are not separated by white spaces • We have to deal consistently with URLs, acronyms, etc. • http://example.com, 2010-09-24, U.S.A. • We have to deal consistently with compound words • hostname, host-name, host name • Solution depends on the language and the domain. • Naive solution: split by white spaces and punctuation

  28. Normalization: Strings Problem: We might extract strings that differ only slightly and mean the same thing. Solution: Normalize strings, i.e., convert strings that mean the same to one common form • Lowercasing, i.e., converting all characters to lower case May be too strong: “President Bush” == “president bush” • Removing accents and umlauts • résumé resume, UniversitätUniversitaet • Normalizing abbreviations • U.S.A.  USA, US  USA

  29. Normalization: Literals Problem: We might extract different literals (numbers, dates, etc.) that mean the same. Solution: Normalize the literals 1.67m 1.67 meters 167 cm 6 feet 5 inches 3 feet 2 toenails 08/01/35 01/08/35 8th Jan. 1935 January 8th, 1935 ... 1.67m 1935-01-08

  30. Information Extraction and beyond Ontological Information Extraction Information Extraction (IE) is the process of extracting structured information (e.g., database tables) from unstructured machine-readable documents (e.g., Web documents). Fact Extraction Instance Extraction Named Entity Recognition ✓ ✓ Tokenization& Normalization ...married Elvis on 1967-05-01 Source Selection 05/01/67  1967-05-01 ?

  31. Named Entity Recognition Named Entity Recognition (NER) is the process of finding entities (people, cities, organizations, ...) in a text. Elvis Presley was born in 1935 in East Tupelo, Mississippi. • We can extract different types of entities: • Entities for which we have an exhaustive dictionary (closed set extraction) ... in Tupelo, Mississippi, but ... States of the USA Countries of the World (?) ... while Germany and France were opposed to a 3rd World War, ... May not always be trivial... ... was a great fan of France Gall, whose songs...

  32. Named Entity Recognition Named Entity Recognition (NER) is the process of finding entities (people, cities, organizations, ...) in a text. Elvis Presley was born in 1935 in East Tupelo, Mississippi. • We can extract different types of entities: • Entities for which we have an exhaustive dictionary (closed set extraction) • Proper names (open set extraction) ... together with the software engineer Bob “the coder” Miller... People ... The region of Northern Urzykistan has been at war with Southern Urzykistan ever since 1208, when... Locations ... BrightFridge Inc. presented their new product, the self-reloading fridge, at this year’s exposition in Paris... Organizations

  33. Named Entity Recognition Named Entity Recognition (NER) is the process of finding entities (people, cities, organizations, ...) in a text. Elvis Presley was born in 1935 in East Tupelo, Mississippi. • We can extract different types of entities: • Entities for which we have an exhaustive dictionary (closed set extraction) • Proper names (open set extraction) • Entities that follow a certain pattern ... was born in 1935. His mother... ... started playing guitar in 1937, when... ... had his first concert in 1939, although... Years (4 digit numbers) Office: 01 23 45 67 89 Mobile: 06 19 35 01 08 Home: 09 77 12 94 65 Phone numbers (groups of digits)

  34. NER: Patterns A pattern is a string that generalizes a set of strings. sequences of the letter ‘a’ a+ ‘a’, followed by ‘b’s ab+ aa abbbbbb abbbb aaaaaa a ab aaaa abbb aaaaaaa digit 0|1|2|3|4|5|6|7|8|9 sequence of digits (0|1|2|3|4|5|6|7|8|9)+ 2 9 6 6543 1 0 4 987 7 5321 5 3 5643 8 => Let’s find a systematic way of expressing patterns

  35. NER: Regular Expressions A regular expression (regex) over a set of symbols Σ is: 1. the empty string 2. or the string consisting of an element of Σ(a single character) 3. or the string AB where A and B are regular expressions (concatenation) 4. or a string of the form (A|B), where A and B are regular expressions (alternation) 5. or a string of the form (A)*, where A is a regular expression (Kleene star) For example, with Σ={a,b}, the following strings are regular expressions: (a|b) ab a aba b

  36. NER: Regular Expressions • Matching • a string matches a regex of a single character • if the string consists of just that character a b  regular expression  matching string • a string matches a regular expression of the form (A)* • if it consists of zero or more parts that match A b a  regular expression (a)*  matching strings a aa aaaaa aaaaa

  37. NER: Regular Expressions • Matching • a string matches a regex of the form (A|B) • if it matches either A or B (a|(b)*) (a|b)  regular expression bbbb b  matching strings bb a a • a string matches a regular expression of the form AB • if it consists of two parts, where the first part matches A • and the second part matches B b(a)* ab  regular expression  matching strings baa ab baaaaa b

  38. NER: Regular Expressions • Given an ordered set of symbols Σ, we define • [x-y] for two symbols x and y, x<y, to be the alternation • x|...|y (meaning: any of the symbols in the range) [0-9] = 0|1|2|3|4|5|6|7|8|9 • A+ for a regex A to be • A(A)* (meaning: one or more A’s) [0-9]+ = [0-9][0-9]* • A{x,y} for a regex A and integers x<y to be • A...A|A...A|A...A|...|A...A (meaning: x to y A’s) f{4,6} = ffff|fffff|ffffff • A? for a regex A to be • (|A) (meaning: an optional A) ab? = a(|b) • . to be an arbitrary symbol from Σ

  39. NER: Regular Expressions A | B Either A or B A* Zero or more occurrences of A A+ One or more occurrences of A A{x,y} x to y occurrences of A A? an optional A [a-z] One of the characters in the range . An arbitrary symbol Example A digit Numbers in scientific format A digit or a letter HTML attributes A sequence of 8 digits Dates 5 pairs of digits, separated by space 5 pairs of digits, separated by a space or a hyphen

  40. NER: Regular Expressions A regex can be matched efficiently by a Finite State Machine (Finite State Automaton, FSA, FSM) • A FSM is a quintuple of • A set Σ of symbols (the alphabet) • A set S of states • An initial state, s0ε S • A state transition functionδ:S xΣ S • A set of accepting states F < S Regex: ab*c c a s1 s3 s0 b Accepting states usually depicted with double ring. Implicitly: All unmentioned inputs go to some artificial failure state

  41. NER: Regular Expressions • A FSM accepts an input string, if there exists a sequence of states, such that • it starts with the start state • it ends with an accepting state • the i-th state, si, is followed by the state δ(si,input.charAt(i)) Sample inputs: abbbc ac aabbbc def Regex: ab*c c a s1 s3 s0 b

  42. NER: Regular Expressions • A non-deterministic FSM has a transition function that maps to a set of states. • A FSM accepts an input string, if there exists a sequence of states, such that • it starts with the start state • it ends with an accepting state • the i-th state, si, is followed by a state in the set δ(si,input.charAt(i)) FSMs can be transformed and simplified while maintaining equivalence, in particular, every non-deterministic FSM can be made deterministic Sample inputs: abbbc ab abc Regex: ab*c|ab c a s1 s3 s0 a b b s4

  43. NER: Regular Expressions A | B Either A or B A* Zero or more occurrences of A A+ One or more occurrences of A A{x,y} x to y occurrences of A A? an optional A [a-z] One of the characters in the range . An arbitrary symbol • Regular expressions • can express a wide range of patterns • can be matched efficiently • are employed in a wide variety of applications • (e.g., in text editors, NER systems, normalization, UNIX grep tool etc.) • Input: • Manual design of the regex • Condition: • Entities follow a syntactic pattern

  44. NER: Sliding Windows Alright, what if we do not want to specify regexes by hand? Use sliding windows... Information Extraction: Tuesday 10:00 am, Rm 407b For each position, ask: Is the current window a named entity? Window size = 1

  45. NER: Sliding Windows Alright, what if we do not want to specify regexes by hand? Use sliding windows of different sizes Information Extraction: Tuesday 10:00 am, Rm 407b For each position, ask: Is the current window a named entity? Window size = 2

  46. NER: Sliding Windows Information Extraction: Tuesday 10:00 am, Rm 407b Prefix window Content window Postfix window • Choose certain features (properties) of windows that could be important: • window contains colon, comma, or digits • window contains week day, or certain other words • window starts with lowercase letter • window contains only lowercase letters • ...

  47. NER: Sliding Windows Information Extraction: Tuesday 10:00 am, Rm 407b Prefix window Content window Postfix window Prefix colon 1 Prefix comma 0 ... ... Content colon 1 Content comma 0 ... ... Postfix colon 0 Postfix comma 1 ... ... The feature vector represents the presence or absence of features of one content window (and its prefix window and postfix window) Feature Vector Features

  48. NER: Sliding Windows Now, we need a corpus (set of documents) in which the entities of interest have been manually labeled. time location NLP class: Wednesday, 7:30amand Thursday all day, room 667 From this corpus, we can compute a set of feature vectors with labels: 1 0 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 0 0 1 1 0 1 1 0 1 0 1 0 1 1 ... ... ... ... Nothing Nothing Time Nothing Location Label:

  49. NER: Sliding Windows Information Extraction: Tuesday 10:00 am, Rm 407b Use the labeled feature vectors as training data for Machine Learning 1 0 0 0 1 1 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 1 Result classify Time Machine Learning (go to the other course to see what that is) Nothing Location

  50. NER: Sliding Windows The Sliding Windows Technique can be used for Named Entity Recognition for nearly arbitrary entities • Input: • a labeled corpus • a set of features • The features can be • arbitrarily complex and • the result depends a • lot on this choice • Condition: • The entities share some • syntactic similarities The technique can be refined by using better features, taking into account more of the context (not just prefix postfix) and using advanced Machine Learning techniques (HMMs, CRFs,...).

More Related