Continual Learning with Gated Incremental Memories for sequential data processing

1. Continual Learning with Gated Incremental Memories for sequential data processing Andrea Cossu, Antonio Carta, Davide Bacciu @ IJCNN 2020

2. Andrea Cossu → andrea.cossu@sns.it → https://andreacossu.github.io/ Ph.D. student in Data Science @ Scuola Normale Superiore @ University of Pisa CIML group @ University of Pisa Department of Computer Science, room 299, Pisa.

3. Sequential data processing CL methods address mainly Computer Vision or Reinforcement Learning tasks What about… NLP? Bioinformatics? Signal processing? Dealing with sequences! http://media.gettyimages.com/ https://imgur.com/v34sAaT https://static1.squarespace.com

4. Recurrent Neural Networks LMN & LSTM https://commons.wikimedia.org/

5. Gated Incremental Memories One module for each distribution (plasticity) Freeze previous module’s parameters after training (stability) Connections between modules to enable knowledge transfer Each module is an expert of its own domain Progressive networks approach

6. Gating Autoencoders LSTM autoencoders (seq2seq model) Autonomously recognize input distribution One autoencoder for each module, trained jointly to the augmented architecture At inference time, the module associated to the autoencoder obtaining the minimum reconstruction loss is used to produce the final output

7. Experiments Simulate sequence learning over images (long sequences!) ● Split MNIST ● Split Devanagari Towards real-world sequential data processing ● Audioset

8. Split MNIST

9. Split Devanagari

10. Audioset

11. Recap + Forgetting can be largely mitigated by the use of expert modules + No need to have (sub)task labels at test time + Model agnostic approach (can be instantiated on any RNN) - Linear cost in the number of tasks (vs. quadratic of Progressive) - No strong guarantees on autoencoders reliability - Need to find more complex tasks to measure transfer

12. EWC and RNNs Weight importance estimated by (an approximation of) the diagonal of the Fisher Information Matrix What about RNNs? → exploding / vanishing gradient (squared!) ● The estimator does not approximate the “true importance” anymore This occurs for every approach in which the weight importance is gradient-based! How to fix this problem? Is it possible to fix it?

13. References [1] A. Cossu, A. Carta, and D. Bacciu, “Continual Learning with Gated Incremental Memories for sequential data processing,” Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN 2020). http://arxiv.org/abs/2004.04077 [2] A. Rusu et al., “Progressive Neural Networks”. http://arxiv.org/abs/1606.04671 [3] D. Bacciu, A. Carta, and A. Sperduti, “Linear Memory Networks,” in Proceedings of the 28th International Conference on Artificial Neural Networks (ICANN 2019). https://arxiv.org/abs/1811.03356