1 / 11

Learning From Measurements in Exponential Families

Learning From Measurements in Exponential Families. Percy Liang, Michael I. Jordan and Dan Klein ICML 2009. Presented by Haojun Chen. Images in these slides are from Percy Liang’s paper and slides. Motivation. Problem:

pmathis
Download Presentation

Learning From Measurements in Exponential Families

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning From Measurements in Exponential Families Percy Liang, Michael I. Jordan and Dan Klein ICML 2009 Presented by Haojun Chen Images in these slides are from Percy Liang’s paper and slides

  2. Motivation • Problem: Given a set of unlabeled examples and a model, one can either label some examples or impose general constraints to provide information to learn the predicator of the model. • Example: Craigslist advertisement • Measurements is introduced to provide a unified framework for integrating both labels and constraints in a coherent manner.

  3. Measurements • : a sequence of input • : corresponding hidden output • Measurement values:

  4. Measurement Examples • Fully-labeled example: To represent the output of , let the components of include Example: • Labeled predicate: For sequence labeling tasks, if input is , we want to know the frequency of some label overall positions. The measurements are where is the length of the sequence. Example:

  5. From Measurements to Model • Goal: learn a predictor based on observed measurements • Predictor: conditional exponential families e.g., linear regression, logistic regression and conditional random field • Exponential families definition: • Bayesian Model:

  6. Approximate Inference • Variational formulation: where • Objective function: • Algorithm: Take alternating stochastic gradient steps

  7. Craigslist Results • Data: 1000 advertisements, 11 possible labels • Measurements: • Fully-labeled examples • Label predicate • Model: Linear-chain conditional random field (CRF)

  8. Active Measurement Selection • Utility of measurement : • Best subsequent measurement: where

  9. Active Learning Algorithm • Define • Algorithm “…, the full algorithm does come with a significant computational cost,…”

  10. Toy Data Results • Input space: • Output space: • Measurements: • Fully-labeled examples • Label predicate • Start with 100 unlabeled data and test on 1000 data

  11. Part-of-speech Tagging Results • Data: 1000 sentences, 45 possible labels • Measurements: • Fully-labeled examples • Label predicate • Model: Independent logistic regression

More Related