1 / 16

Short Term Memories and Forcing the Re-use of Knowledge for Generalization

Short Term Memories and Forcing the Re-use of Knowledge for Generalization. Laurent Orseau INSA/IRISA Rennes, France. Neural Networks. Good learning devices for many interesting problems toward generalization

avian
Download Presentation

Short Term Memories and Forcing the Re-use of Knowledge for Generalization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Short Term Memories and Forcing the Re-use of Knowledge for Generalization Laurent Orseau INSA/IRISA Rennes, France

  2. Neural Networks • Good learning devices for many interesting problems toward generalization • In theory, RNNs are equivalent to Turing Machines for representation, but learning is not the same problem

  3. Marcus et al. Task (1999) • Surface, explicit sequence: ABCDE • E follows ABCD, not another letter • Abstraction: AAB, BBD, EEA, … (≠ABA, BCB, …) • Repetition is important, not explicit symbols • Infants can learn it • (R)NNs cannot! • One solution (Dominey et al.): Add special Short Term Memories (STMs) to a Temporal Recurrent Network

  4. Only “Mere” Abstraction? • STMs: abstraction  generalization? • Refine Dominey’s STMs • Use the temporal framework, even for static tasks • Force the re-use of knowledge • Experiment on a general classification task

  5. Short Term Memories • STM #d activated if any input activated a second time after d time steps • Seq. AA (or BB, etc.) activates STM #1 • Seq. BAB activates STM #2 • Seq. ABCA activates STM #3 • etc.

  6. Architecture (1) • Simple TDNN: • k input sets • Input set i-1 = copy of input set i at t-1 • STMs on inputs; • k STMs/input set, each for a different delay • automatically updated (internal)

  7. Architecture (2) • External loop (no learning) • Agent “hears” what it says, “thinks aloud”

  8. External Loop • Feed forward: agent does not hear what it says • External loop: agent says ABCDE agent hears ABCDE • + Actions and inputs are merged: teacher says ABCDE agent hears ABCDE  STMs used for both: copy or recognition of repetition • Supervision with reinforcements

  9. Action Selection & Learning • Action selection: • Each input-action tested • One with best predicted reinforcement chosen • If agent must say input-action a: • If it does say a: teacher rewards it • If it says another letter: teacher punishes it • If it says nothing: teacher says a in the agent’s place and rewards it

  10. Forcing the Re-use of Knowledge • The teacher uses the loop to make the agent re-use its own knowledge  Auto-stimulation • Ex 1: • Knowing: ABC, ABCD, CDE • Teacher: AB Agent: C Hears: ABC (loop) • Agent: D Hears: ABCD • Agent: E • By saying AB, the teacher forces CDE

  11. Classification Task abcde | fghij | klmno | pqrst | uvwxy | z A | B | C | D | E | F • Task1 (rote learning): • What is the group of m? C • But does not know what A contains, etc. • Task2, knowing Task1: • Is m in the group C? yes • Seems simple for human, but not for NNs alone • Where is the need for abstraction?

  12. Learning Task1 • rote learning, no generalization needed. • Teacher: gpa Agent: a Hears: gpaa • Teacher: gpb Agent: a Hears: gpba • … • Teacher: gpz Agent: f Hears: gpzf • gp: • Name of the problem • Needed to be re-used

  13. Re-use STM #4 teacher isfb isfb f b b y (yes) isfcgpfb n (no) STM #4 STM #4 Learning Task2 (1) isfb g p f Or

  14. Learning Task2 (2) • Training on: • Letters: a.. j, l, n, p, r, s, t • Groups: A, B, C, D • Must generalize to all letters and groups! • (R)NNs alone: could do only clustering,no generalization to unseen groups

  15. Parameters • TDNN: • Inputs/actions: 26 letters of the alphabet • k=4: 4 input sets, 4 STMs/input set • Output: reinforcement • Must learn to stay quiet apart from what is provided

  16. Results • Task1: • 8 hidden neurons • learned perfectly (no generalization) • Task2: • weights frozen • 5 neurons added • Perfect generalization: • Seen groups, unseen letters • But also unseen groups, unseen letters • Not possible without STMs

More Related