Understanding Neural Networks: Depth and Computational Graphs

3. Objectives for Lecture 8: Neural Networks Depth of understanding • After the lecture you are able to…

5. Repetition

6. Repetition

7. Last Week: Today:

9. Computational Graph Options: 1. Analytical 2. Differencequotient 3. Computationalgraph

10. Computational GraphSimple example Letscalculatesome derivatives!

11. Computational GraphSimple example Letscalculatesome derivatives!

12. Computational GraphSimple example Letscalculatesome derivatives!

13. Computational GraphSimple example Letscalculatesome derivatives!

14. Computational GraphSimple example Letscalculatesome derivatives!

15. Computational GraphSimple example Letscalculatesome derivatives!

16. Computational GraphSimple example Letscalculatesome derivatives!

17. Computational GraphSimple example Letscalculatesome derivatives!

18. Computational GraphSimple example Gradient „flows“ Upstream Gradient Downstream Gradient Letscalculatesome derivatives!

19. Computational GraphSimple example Letscalculatesome derivatives!

20. Computational GraphSimple example Letscalculatesome derivatives!

21. Computational GraphStandard operations Multiplication Addition: Downstream gradient remainsthe same Downstream gradient switchestootherfactor

22. Computational GraphSimple example, withnumbers

23. Computational GraphSimple example, withnumbers

24. Computational GraphSimple example, withnumbers

25. Computational GraphSimple example, withnumbers

26. Computational GraphSimple example, withnumbers

27. Computational GraphSimple example, withnumbers

28. Computational GraphAbstract function

29. Computational GraphAbstract function Forward path

30. Computational GraphAbstract function Savinglocalgradientsduringforwardpath Forward path

31. Computational GraphAbstract function Alreadycalculatedduringforwardpath Backwardpath

32. ComputationalGraphAbstract function Alreadycalculatedduringforwardpath Backwardpath

33. ComputationalGraphAbstract function

34. Computational Graph ComplexExample

35. ComputationalGraphComplexExample

36. ComputationalGraphComplexExample

37. ComputationalGraphComplexExample

38. Computational Graph Neuron

39. Computational GraphNeuron

40. Computational GraphNeuron

41. Computational GraphNeuron Local derivative

42. Computational GraphNeuron

44. BackpropagationNeuralChain Forward path Input Neuron Layer 1 Layer 2 Loss Function

45. BackpropagationNeural Chain Forward path Input Neuron Layer 1 Layer 2 Loss Function

46. BackpropagationNeuralChain Forward path Input Neuron Layer 1 Layer 2 Loss Function Backwardpath

47. BackpropagationNeuralChain

48. BackpropagationNeuralChain Forward path Input Neuron Layer 1 Layer 2 Loss Function Backwardpath

49. BackpropagationNeuralChain

50. BackpropagationNeuralChain Forward path Input Neuron Layer 1 Layer 2 Loss Function Backwardpath