180 likes | 344 Views
Using MCMC. Separating MCMC from Bayesian Inference? Line fitting revisited A toy equaliser problem Some lessons A problem in film restoration/retouching. Articulate Probabilities [Bayesian Inference]. Try to see if you can integrate out nuisances. Derive the Posterior.
E N D
Using MCMC • Separating MCMC from Bayesian Inference? • Line fitting revisited • A toy equaliser problem • Some lessons • A problem in film restoration/retouching
Articulate Probabilities [Bayesian Inference] Try to see if you can integrate out nuisances Derive the Posterior MCMC is (just) a tool Choose a Model Identify Parameters Need better model If solution not ok Use MCMC Solve Deterministically Direct, CG, Steepest Descent etc Manipulate Random Samples To get one answer (if you want) Gives you one answer
Can always design single parameter-at-a-time schemes. So iterations can be very low complexity Simple iterations = long convergence Gives you a picture of alternate answers Do you really need alternate answers? Will always allow you to get to “best” solution Iterations can be high complexity? Convergence can be rapid (e.g. CG) for well defined problems Gives you just one answer Can give local minimum for non-linear problems Good, Bad Others MCMC
Ugly • To solve your problem you need a good model • MCMC is not really going to help you if you have the wrong model • MCMC suited to BIG problems: but what is BIG really? • E.g. Exhaustive search for motion estimation is possible in real time (TV rates) in hardware: why bother with other things? (an exaggeration … but interesting nevertheless)
Line Fitting (again) Needs Latex Observed Data Actual Line Initial Guess
Typical Results See Matlab demo Nice Convergence because we can draw samples directly
Typical Results c m var_e
Watch out • All random number generators are not created equal • (See NR) • Harder problems require longer runs (of course) • Sometimes hard to get all bugs out because its all a random search anyway
Blind (?) Equalisation Noise Signal 2nd Order All pole System Rec’d Signal Identify the system coefficients AND recover the original signal Comms, Deblurring, Overshoot Cancellation
Equaliser Problem Now more latex
Direct numerical sampling P(1) = 0.3, p(2) = 0.25, P(3) = 0.2, p(4) = 0.25 0 1 0.3 2 0.25 3 0.2 0.75 4 0.25 1 Number line interpretation 71 points evaluated
Gibbs sampler 1 (equaliser1.m) Back to Latex Typical Estimated System Actual System X 20 ! 300 iterations
Gibbs Sampler II (equaliser2.m)Using Filter Bank (system choices) 30 filters
Lessons • Gibbs sampler takes big problems and breaks them into lots of small ones • Spotting the functional form of a known p.d.f. is a useful skill. Books help. • If all else fails, can always sample directly • MCMC does not necessarily solve your problem. Good priors, better models are still important • Deterministic/Stochastic Hybrid mix is v. useful