GLOW- Global and Local Algorithms for Disambiguation to Wikipedia

1. GLOW- Global and Local Algorithms for Disambiguation to Wikipedia Lev Ratinov, Doug Downey, Mike Anderson, Dan Roth • Problem Definition: • “Find all the “interesting” concepts in the input text, and disambiguate them to the corresponding Wikipedia titles”. • Problem Formulation: • Γ* is a solution to the problem, a set of mention-title pairs (m,t). • Evaluate the localmatching quality using Φ(m,t). • Evaluate the globalstructure based on (1) pair-wise coherence scores Ψ(ti,tj) (2) a solution-proxy Γ’ (Γ’ can be unambiguous mentions (such as Jiangsu), or local solutions to all mentions). Γ’ allows solving the problem locally while taking into account the global structure.) End-to-end system. experimental results: GLOW Vs. Previous State of the Art (Milne 08) It’s a version of Chicago – the standard classic Macintosh menu font, with that distinctive thick diagonal in the ”N”. Chicago was used by default for Mac menus through MacOS 7.6, and OS 8 was released mid-1997.. ChicagoVIIIwas one of the early 70s-era Chicago albums to catch my ear, along with Chicago II. • Local Match Scorers: • P(title|surface form) : Baseline • PrTFIDF(title text | input document) • PrTFIDF(title context | input document) • PrTFIDF(title text | surface form context) • PrTFIDF(title context | surface form context) • PrTFIDF is a probabilistic version of TFIDF (Joachims 97) • Global Structure Scorers: • Average, Minimal, Maximal relatedness of the title to: • unambiguous mentions. • baseline disambiguation of the NER mentions. • baseline disambiguation of all the mentions. • Using PMI, Normalized Google Distance. Visit our demo: http://cogcomp.cs.illinois.edu/demo/wikify/ • Discussion: • Table T1 summarizes dataset statistics. We fail to detect some mentions. We are only investigating “identifiable mentions” here. • Figure F1 shows that P(title|surface form) is a very strong baseline. Choosing the link based on this decision alone leads to over 80% accuracy on all datasets except MSNBC. We add disambiguation candidates based on P(title|surface form). After 5 candidates, adding more candidates is inefficient. We cut off candidate generation at 20. If the correct solution appears in top 20 candidates, then the mention is “solvable”. • Table T2 compares ranking accuracy with different strategies on “solvable mentions”. We see that the global approaches are better, and combining local and global leads to minor improvements. • It is important, however, to identify “non-solvable mentions”. For example, we want to know that “Michael Rush, a 16-year-old hunter” does not have a Wikipedia page. We train a linker, a classifier for that task and test its performance in Table T3. It shows that the local features are better for this task, but the end performance is brittle. Identifying names that do not appear in Wikipedia is currently unsolved problem. F1: Disambiguation Candidates Generation T2: Ranking Accuracy on “Solvable Mentions” Combining local and global matchers: Font or City ? ”Chicago was used by default for Mac menus through MacOS 7.6, and OS 8 was released mid-1997..” T1: Datasets T3: Impact of Training a Linker on End Performance We train an SVM ranker which makes use of the difference between the feature vectors. It abstracts over the text and learns which signals to trust more: SVM(font Vs. city)=(0.99-0.001,0.001-0.01,0.03-0.23,0-0.004) This research is sponsored by DARPA Machine Reading Program under Air Force Research Laboratory (AFRL) prime contract No. FA8750-09-C-0181 and by the Army Research Laboratory (ARL) under agreement W911NF-09-2-0053.