1 / 39

Math Primer

Math Primer. Outline. Calculus: derivatives, chain rule, gradient descent, taylor expansions Bayes Rule Fourier Transform Dynamical linear systems. Calculus. Derivatives Derivative=slope. Calculus. Derivative: a few common functions (x n )’=nx n-1 (x -1 )’=-1/x 2 = x -2

elwyn
Download Presentation

Math Primer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Math Primer

  2. Outline • Calculus: derivatives, chain rule, gradient descent, taylor expansions • Bayes Rule • Fourier Transform • Dynamical linear systems

  3. Calculus • Derivatives • Derivative=slope

  4. Calculus • Derivative: a few common functions • (xn)’=nxn-1 • (x-1)’=-1/x2 = x-2 • exp(x)’=exp(x) • log(x)’=1/x • cos(x)’=-sin(x)

  5. Calculus • Derivative: Chain rule • Ex: gaussian • z=h(x)=exp(-x2)’,f(y)=exp(y), y=g(x)=-x2 • f’(y)=exp(y)= exp(-x2), g’(x)=-2x • z’= f’(y)g’(x) =exp(-x2)(-2x)

  6. Calculus • Finding minima: gradient descent f’(x0) < 0 f(x) dx > 0 f’(x*)=0 x0+dx x0 x* dx = -a f’(x0)

  7. Calculus • Example: minimizing an error function

  8. Calculus • Taylor expansion

  9. Calculus

  10. Bayes rule • Example: drawing from 2 boxes • 2 boxes (B1,B2) • P(B1)=0.2,P(B2)=0.8 (Prior) • Balls with two colors (R,G) • B1=(16R,8G), B2=(8R,16G) • P(R|B1)=2/3, P(G|B1)=1/3 (Conditional) • P(R|B2)=1/3, P(G|B2)=2/3 (Conditional)

  11. Bayes rule • Joint distributions • P(G,B1)=P(B1)P(G|B1)=0.2*0.33=0.066 • P(G,B2)=P(B2)P(G|B2)=0.8*0.66=0.528 • P(X,Y)=P(X|Y)P(Y) • P(Y,X)=P(Y|X)P(X) • P(Y|X)P(X)=P(X|Y)P(Y)

  12. How do you get this? Marginalize Bayes rule • Bayes rule • P(Y|X)P(X)=P(X|Y)P(Y) • P(Y|X)=P(X|Y)P(Y)/P(X) • If you draw G, what is the probability that it came from box1? • P(B1|G)=P(G|B1)P(B1)/P(G)

  13. Bayes rule Marginalization • P(G)=P(G,B1)+P(G,B2)=0.066+0.528=0.6 • P(G)=P(G|B1)P(B1)+P(G|B2)P(B2) • P(Y)=Sx P(Y,X) • P(Y)=Sx P(Y|X)P(X) • P(Y|X)=P(X|Y)P(Y)/SYP(X|Y)P(Y)

  14. Sum to one Bayes rule • Bayes rule • If you draw G, what is the probability that it came from Box1 or Box2? • P(B1|G)=P(G|B1)P(B1)/P(G) =(0.33*0.2)/0.6=0.11 • P(B2|G)=P(G|B2)P(B2)/P(G) =(0.66*0.8)/0.6=0.89

  15. Bayes rule • P(A,B|C)=P(A|B,C)P(B|C) • P(B|A,C)=P(A|B,C)P(B|C)/P(A|C)

  16. Fourier transform • Basis in linear algebra • Basis function: dirac • Basis function: sin

  17. Fourier Transform • Decomposition in sum of sin and cosine • Power: first term is the DC • Phase • Fourier transform for Dirac Sin Gaussian (inverse relationship)

  18. Fourier Transform • Convolution and products

  19. Fourier transform • Fourier transform of a Gabor

  20. Fourier transform • Eigenspace for liner dynamical system…

  21. Dynamical systems • Stable if l<0, unstable otherwise

  22. Dynamical systems Fixed Point

  23. Dynamical systems Stable if f’(x0)<0, unstable otherwise.

  24. Dynamical systems

  25. Dynamical systems

  26. Dynamical systems • go into eigen space • Equations decouple Stable ifl1<0, unstable otherwise.

  27. Dynamical systems

  28. Dynamical systems • go into eigen space • Equations decouple

  29. Dynamical systems • go into eigen space • Equations decouple Stable is f’(x0)<0, unstable otherwise.

  30. Dynamical systems • Fixed point • Saddle point • Unstable point • Stable and unstable oscillations: complex eigenvalues

  31. Nonlinear Networks • Discrete case: Stable if |l|<1, unstable otherwise

  32. Nonlinear Networks • Discrete case:

  33. Nonlinear Networks • Dynamics around attractor:

  34. Nonlinear Networks • Stable Fixed point: |l1|<1, |l2|<1

  35. Nonlinear Networks • Saddle Point: |l1|>1, |l2|<1

  36. Nonlinear Networks • Unstable Fixed point: |l1|>1, |l2|>1

  37. Nonlinear Networks • Line Attractor: l1=1, |l2|<1

  38. Nonlinear Networks • Oscillation: complex l’s

  39. Nonlinear Networks: global stability • Lyapunov Function: function of the state of the system which is bounded below and goes down over time. If such a function exists, the system is globally stable. • Ex: Hopfield network, Cohen-Grossberg network

More Related