1 / 71

Teaching Machines to Learn by Metaphors

Teaching Machines to Learn by Metaphors. Omer Levy & Shaul Markovitch Technion – Israel Institute of Technology. Concept Learning by Induction. Few Examples. Transfer Learning. Target (New). Source (Original). Define: Related Concept. Transfer Learning Approaches.

zeki
Download Presentation

Teaching Machines to Learn by Metaphors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Teaching Machines to Learn by Metaphors Omer Levy & ShaulMarkovitch Technion – Israel Institute of Technology

  2. Concept Learning by Induction

  3. Few Examples

  4. Transfer Learning Target (New) Source (Original)

  5. Define: Related Concept

  6. Transfer Learning Approaches • Common Inductive Bias • Common Instances • Common Features

  7. Different Feature Space

  8. Example -3 -2 0 2 3

  9. Example -3 -2 0 2 3 0 4 9

  10. Example -3 -2 0 2 3 0 4 9

  11. Common Inductive Bias -3 -2 0 2 3 0 4 9

  12. Common Inductive Bias -3 -2 0 2 3 0 4 9

  13. Common Instances -3 -2 0 2 3 0 4 9

  14. Common Features 3 2 4 9 -2 -3

  15. New Approach to Transfer Learning

  16. Our Solution: Metaphors

  17. Metaphors Target (New) Source (Original)

  18. Source Concept Learner Target Metaphor Learner +/-

  19. is a perfect metaphor if: • is label preserving • is distribution preserving

  20. Theorem If is a perfect metaphor - and - is a source hypothesis with error - then - is a target hypothesis with error

  21. The Metaphor Theorem If is an -perfect metaphor - and - is a source hypothesis with error - then - is a target hypothesis with error

  22. Redefine Transfer Learning Given source and target datasets, find a target hypothesis such that is as small as possible.

  23. Redefine Transfer Learning Given source and target datasets, find an -perfect metaphor such that is as small as possible.

  24. Metaphor Learning Framework

  25. Concept Learning Framework Search Algorithm Hypothesis Space Data Evaluation Function

  26. Metaphor Learning Framework Source Search Algorithm Metaphor Space Target Evaluation Function

  27. Metaphor Evaluation

  28. Metaphor Evaluation • is label preserving • is distribution preserving

  29. Metaphor Evaluation • is label preserving Empirical error over target dataset • is distribution preserving Statistical distance between and

  30. Metaphor Evaluation

  31. Metaphor Evaluation

  32. Metaphor Evaluation

  33. Metaphor Evaluation

  34. Metaphor Spaces

  35. Metaphor Spaces • General • Few Degrees of Freedom • Representation-Specific Bias

  36. Geometric Transformations Я R

  37. Dictionary-Based Metaphors cheese queso

  38. Linear Transformations

  39. Which metaphor space should I use?

  40. Automatic Selection of Metaphor Spaces Which metaphor space should I use?

  41. Occam’s Razor Which metaphor space should I use? Automatic Selection of Metaphor Spaces

  42. Structural Risk Minimization Which metaphor space should I use? Automatic Selection of Metaphor Spaces Occam’s Razor

  43. Automatic Selection of Metaphor Spaces

  44. Automatic Selection of Metaphor Spaces

  45. Automatic Selection of Metaphor Spaces

  46. Empirical Evaluation

  47. Reference Methods Baseline • Target Only • Identity Metaphor • Merge State-of-the-Art • Frustratingly Easy Domain Adaptation • Daumé, 2007 • MultiTask Learning • Caruana, 1997; Silver et al, 2010 • TrAdaBoost • Dai et al, 2007

  48. Digits: Negative Image

More Related