1 / 15

Lecture 9

Lecture 9. State Space Gradient Descent Gibbs Sampler with Simulated Annealing. Continuous Space Gradient Descent. E (a, b, c). Step size l has to be small 0.01 might not work 0.00001 begins to work. State Space Gradient Descent. For each pixel x,y For each possible state S

walker
Download Presentation

Lecture 9

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 9 State Space Gradient Descent Gibbs Sampler with Simulated Annealing

  2. Continuous Space Gradient Descent E (a, b, c) Step size l has to be small 0.01 might not work 0.00001 begins to work

  3. State Space Gradient Descent For each pixel x,y For each possible state S if F(x,y) = 0, E0 = ... if F(x,y) = 1, E1 = ... . . if F(x,y) = 255, E255 = … Select state with minimum E Let F(x,y) = S Repeat Until no change in E

  4. State Space Gradient Descent F0(x,y) F1(x,y) RANDOM

  5. x,y-1 x-1,y x,y Energy means Global Energy Change in (x,y) = change in E of (x,y), (x-1,y), (x,y-1) Clique - neighborhood affected

  6. State Space Gradient Descent (x-1,y-1) (x,y-1) (x-1,y) (x,y)

  7. State Space Gradient Descent F0(x,y) E0(x,y) E1(x,y) F1(x,y) Ft(x,y)

  8. State Space Gradient Descent Greedy - Takes best step Problem - Stuck at local minimum

  9. Gradient Descent Algorithm 1. Initialize F0(x,y) = Random 0..255 2. For each pixel (x,y) For each state S = 0..255 if F(x,y) = 0, E0 = ... if F(x,y) = 1, E1 = ... . . if F(x,y) = 255, E255 = … Choose state with minimum E, F(x,y) = S 3.Repeat step 2 until E is stable (not decrease)

  10. Gibbs Sampler 1. Start Temperature T is high 2. Initialize F0(x,y) = Random 0..255 3. For each pixel (x,y) For each state S = 0..255 if F(x,y) = 0; E0 = ... ; P0 = ... if F(x,y) = 1; E1 = … ; P1 = ... . . if F(x,y) = 255; E255 = …; P255 = ... For each Pi = Pi/sum(Pi) 4.Sample for state S from pdf Pi F(x,y) = S 5. Reduce T = T * 0.9 6. Repeat step 3-4 while E is not stable

  11. Gibbs Sampler 3 issues - 1) From E, how to get P 2) How to sample from PDF 3) Why reduce T, what is T for?

  12. Pia E - Energy - 2.718 Gibbs Sampler: Find P proportional normalizing factor Rand(0..1) = .43 F(x,y) = 3

  13. Gibbs Sampler : Why T? T High - Random Walk 20% T Low - Approach o (Gradient Descent)

  14. Gibbs Sampler : Why T? Analyze States = 3 E(0) = 2 E(1) = 3 E(2) = 4 T=100 P = .98 P = .97 P = .96 P = .33 P = .33 P = .33 T=10 P = .82 P = .74 P = .67 P = .36 P = .33 P = .31 T=100 P = 2.06*10-9 P = 9.3*10-14 P = 4.2*10-18 P = .999 P = .00001 P = .000001 T controls distribution of PDF

  15. Why slowly reduce T gets global min?

More Related