1 / 19

Bootstrapping

Bootstrapping. Tom Griffiths. Bootstrapping. How to learn words without knowing words Various proposals: “semantic bootstrapping” (Pinker, 1984) “syntactic bootstrapping” (Gleitman, 1990) Characterized by accelerated learning (e.g. Regier, 2004) Question:

addison-hyde
Download Presentation

Bootstrapping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bootstrapping Tom Griffiths

  2. Bootstrapping • How to learn words without knowing words • Various proposals: • “semantic bootstrapping” (Pinker, 1984) • “syntactic bootstrapping” (Gleitman, 1990) • Characterized by accelerated learning (e.g. Regier, 2004) • Question: • when is bootstrapping possible?

  3. Word learning “blicket” “blicket” “blicket”

  4. Likelihood Prior probability Posterior probability Sum over space of hypotheses Bayes’ theorem h: hypothesis d: data

  5. Bayesian word learning (Tenenbaum, 1999; Tenenbaum & Xu, 2002) • Data • scene-word pairs • Hypotheses • functions labeling scenes • Likelihood • weak sampling • strong sampling x h w

  6. “blicket” p(d|h) = 0

  7. “blicket” p(d|h) = 1/3

  8. “blicket” “blicket” “blicket” p(d|h) = (1/3)3

  9. “blicket” p(d|h) = 1/12

  10. “blicket” “blicket” “blicket” p(d|h) = (1/12)3

  11. Bootstrapping • Bayesian word learning is a form of semantic bootstrapping (Niyogi, 2002) • What about accelerated learning? • non-linear* increase in probability of correct answer for a random scene and word • When can it occur? • not when hypotheses independent and all equally likely, when using weak sampling • speculation: hypotheses are dependent

  12. Forms of dependency • Hierarchical priors • unknowns across learning events • Compositional priors • unknowns within learning events

  13. Hierarchical priors x x x x h h h h w w w w “blicket” “toma” “dax” “wug”

  14. “dax” “blicket” “toma” “wug”?

  15. Hierarchical priors • What is contained in a hierarchical prior? • Any learned information that constrains scene-word mappings • typical referents (whole object) • dimensions of stimuli (shape/substance) • pragmatic dependencies (mutual exclusivity) • sound and meaning (morphology)

  16. h G x h1 x h2 h1 x h2 w1 w2 w1 w2 w1 w2 holistic independent compositional Compositional hypotheses “blicket toma”

  17. Compositional hypotheses • Good news: • express syntactic bootstrapping • model referential uncertainty • Bad news • requires complete linguistic theory

  18. Bootstrapping • When do we see accelerated learning? • speculation: dependent hypotheses • Sources of dependency in language • hierarchical priors • compositional hypotheses • Bootstrapping goes beyond language • learning causal theories aids learn causal relationships, learning concepts…

More Related