Lecture 12: Sequence to sequence models

1. Lecture 12: Sequence to sequence models Alireza Akhavan Pour CLASS.VISION

2. Sequence to sequence model: Introduction and concepts

4. Sequence to sequence model Jane visite l’Afrique en septembre Jane is visiting Africa in September. [Sutskever et al., 2014. Sequence to sequence learning with neural networks] [Cho et al., 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation]

5. Sequence to sequence model Jane visite l’Afrique en septembre Jane is visiting Africa in September. Encoder Decoder [Sutskever et al., 2014. Sequence to sequence learning with neural networks] [Cho et al., 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation]

6. Image captioning MAX-POOL MAX-POOL A cat sitting on a chair MAX-POOL = 3 3 s = 2 3 3 s = 2 5 5 same 3 3 s = 2 3 3 3 3 same 3 3 11 11 s = 4 1313384 1313256 66256 1313384 2727256 55556 272796 1313256 Softmax 1000 4096 4096 9216 [Mao et. al., 2014. Deep captioning with multimodal recurrent neural networks] [Vinyals et. al., 2014. Show and tell: Neural image caption generator] [Karpathy and Li, 2015. Deep visual-semantic alignments for generating image descriptions]

7. Machine translation as building a conditional language model Language model:

8. Machine translation as building a conditional language model Stateی که encoder ایجاد کرده Language model: وکتور 0 Machine translation: Conditional language model

9. Finding the most likely translation Jane visite l’Afrique en septembre. English Jane is visiting Africa in September. French Jane is going to be visiting Africa in September. In September, Jane will visit Africa. Her African friend welcomed Jane in September.

10. Why not a greedy search? Jane is visiting Africa in September. Jane is going to be visiting Africa in September. >

11. Beam search

12. Beam search algorithm B = 3 (Beam width) Step 1 a in jane English French september zulu

13. Beam search algorithm (B=3) in Step 1 Step 2 a a aaron in september in zulu a is jane visiting zulu a september zulu zulu

14. Beam search algorithm (B=3) in Step 1 Step 2 a a aaron in september in zulu a is jane visiting zulu a september zulu zulu

15. Beam search jane in september is jane visits in september jane is jane visits jane visits africa in september. <EOS> برای هر کدام از این 3 خروجی، احتمال ها را نیز ذخیره کرده ایم

16. Refinements to beam search

17. Length normalization |) P( | , وابسته به طول خروجی! ? ?

18. Beam search discussion Beam width B? you might see in the production setting B=10. B=100, B=1000 are uncommon (sometimes used in research settings) Large B: Better result, slower Small B: worse result, faster Unlike exact search algorithms like BFS (Breadth First Search) or DFS (Depth First Search), Beam Search runs faster but is not guaranteed to find exact maximum for.

19. Error analysis on beam search

20. Example Jane visite l’Afrique en septembre. Human: Jane visits Africa in September. Algorithm: Jane visited Africa last September. … visits Jane Africa • RNN • Beam search

21. Error analysis on beam search Human: Jane visits Africa in September. Algorithm: Jane visited Africa last September. Case 1: (P(y* | X) > P(ŷ | X)) Beam search chose . But attains higher Conclusion: Beam search is at fault. Case 2: (P(y* | X) <= P(ŷ | X)) is a better translation than But RNN predicted Conclusion: RNN model is at fault.

22. Error analysis process Algorithm At fault? Human Jane visits Africa in September. Jane visited Africa last September. Figures out what faction of errors are “due to” beam search vs. RNN model

23. منابع https://www.coursera.org/specializations/deep-learning https://towardsdatascience.com/sequence-to-sequence-model-introduction-and-concepts-44d9b41cd42d