1 / 16

Declarative Learning Models for Natural Language Processing

Declarative Learning Models for Natural Language Processing. Aria Haghighi 12/08/2006. Overview. Need Quick NLP System Deployment New languages New domains Typical User Sophisticated engineer Little statistical expertise No time to label data!. Overview. Target Label.

nishi
Download Presentation

Declarative Learning Models for Natural Language Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Declarative Learning ModelsforNatural Language Processing Aria Haghighi 12/08/2006

  2. Overview Need Quick NLP System Deployment • New languages • New domains Typical User • Sophisticated engineer • Little statistical expertise • No time to label data!

  3. Overview Target Label Prototypes Target Label Prototypes Annotated Data Unlabeled Data Prototype List +

  4. Sequence Modeling Tasks Size Restrict Terms Location Features Information Extraction: Classified Ads Newly remodeled2 Bdrms/1 Bath, spacious upper unit, located in Hilltop Mall area. Walking distance to shopping, public transportation, schools and park.Paid water and garbage.No dogs allowed. Newly remodeled 2 Bdrms/1 Bath, spacious upper unit, located in Hilltop Mall area. Walking distance to shopping, public transportation, schools and park. Paid water and garbage. No dogs allowed. Prototype List

  5. Sequence Modeling Tasks PUNC NN VBN CC JJ CD IN DET NNS IN NNP RB English POS Newly remodeled 2Bdrms/1Bath,spacious upper unit,locatedin Hilltop Mallarea.Walkingdistance toshopping, public transportation,schoolsandpark.Paid water andgarbage. Nodogs allowed. Newly remodeled 2 Bdrms/1 Bath, spacious upper unit, located in Hilltop Mall area. Walking distance to shopping, public transportation, schools and park. Paid water and garbage. No dogs allowed. Prototype List

  6. Generalizing Prototypes awitness reported awitness reported said the president • Tie each word to its most similar prototype

  7. Generalizing Prototypes DET NN VBD y: a witness reported x: ‘reported’ & VBD suffix-2=‘ed’ & VBD sim=‘said’ & VBD Weights ‘reported’ Æ VBD = 0.35 suffix-2=‘ed’ Æ VBD= 0.23 sim=‘said’ Æ VBD = 0.35

  8. English POS Experiments • Data • 193K tokens (about 8K sentences) of WSJ portion of Penn Treebank • Features [Smith & Eisner 05] • Trigram tagger • Word type, suffixes up to length 3, contains hyphen, contains digit, initial capitalization

  9. English POS Experiments BASE • Fully Unsupervised • Random initialization • Greedy label remapping

  10. English POS Experiments • Prototype List • 3 prototypes per tag • Automatically extracted by frequency

  11. English POS Distributional Similarity -1 +1 • Judge a word by the company it keeps <s> the president said a downturn is near </s> • Collect context counts from 40M words of WSJ • Similarity [Schuetze 93] • SVD dimensionality reduction • cos() similarity measure

  12. English POS Experiments PROTO+SIM • Add similarity features • Top five most similar prototypes that exceed threshold 67.8% on non-prototype accuracy

  13. English POS Transition Counts Learned Structure Target Structure

  14. Classified Ads Experiments • Data • 100 ads (about 119K tokens) from [Grenager et. al. 05] • Features • Trigram tagger • Word type

  15. Classified Ads Experiments BASE • Fully Unsupervised • Random initialization • Greedy label remapping

  16. Classified Ads Experiments • Prototype List • 3 prototypes per tag • 33 words in total • Automatically extracted by frequency

More Related