Multidimensional space “The Last Frontier”

1. Multidimensional space“The Last Frontier” • Optimization • Expectation • Exhaustive search • Random sampling • “Probabilistic random” sampling Lecture 18, CS567

2. Optimization – “I want the bestest there is” • Problem: Given F(x1, x2, …xn), find argmin{x} F(x1, x2, …xn) where xi = Data AND/OR Model AND/OR Parameters and F = P or some other function, e.g., energy • Ideal goal: Finding global minimum (xor maximum) • “Finding the happiest person in the whole wide world” • “Finding the deepest trench in the ocean” • Ideal approach: Exhaustive search • Guaranteed to find global minimum • Brute force • Subject to resource limits (“Will take a lot of diving!”) Lecture 18, CS567

3. Optimization - Random Sampling • Spectrum of Approaches: • Exhaustive…………………………………………………….Random Sampling • Random sampling: • Find several local minima starting from random points • Pick the lowest minimum as the approximation of the global minimum • Examples: • “Pick a few air travel websites at random, and buy your ticket from the one giving the lowest fare” (finding the cheapest website) • “Pick a few casinos at random, and try your luck at each” (finding the luckiest casino in the world) • Problems • If the number of websites/casinos is very large, a small sample may be far from global minimum • If a large sample is taken, begins to approximate exhaustive search • Example: Monte Carlo algorithm Lecture 18, CS567

4. Optimization • Dynamic programming • Equivalent to exhaustive search without having to use brute force • However, applicable only to problems that satisfy principle of optimality • Gradient descent: Need to combine with some random/iterative element to find global minimum • Backpropagation in NN training: Classic gradient descent • Line search: Update direction only when minimum along a particular direction is reached (“Go down mid-line of saddle before turning towards flank”) • Second derivative based methods • Step size based on second derivative of minimization function, and history of past steps taken • Examples: Newton-Raphson, Conjugate gradient (scales better) • Evolutionary/Genetic algorithms • Simulation based on principles of biological evolution, with cost function of choice • Benefiting from no holds barred multiple inheritance Lecture 18, CS567

5. Optimization – Simulated Annealing • Analogies • Annealing steel to a stable form • Crystallization best with slow evaporation • Using a pogo stick with a bounce that decreases with time, to find lowest valley • Cost function has an additional term that is directly proportional to temperature • Relative importance of this term is progressively decreased • Heat system to high temperature, i.e., give it a lot of energy (all states become roughly equally probable) • Cool system slowly (probability distribution gradually approaches the underlying normal temperature one) • Example: • Producing a 3D structural model of a molecule, given diffraction/NMR data (constraints) Lecture 18, CS567

6. Computing Expectations “How great am I?” • Problem: Given F(x1, x2, …xn), find E[F(x1, x2, …xn)] where xi = Data AND/OR Model AND/OR Parameters and F = P or some other function, e.g., energy • Ideal goal: Finding global average • “What is the average happiness in the whole wide world?” • “What is the average depth of the ocean?” • Ideal approach: Exhaustive search • Guaranteed to find global average • Brute force • Subject to resource limits (“Will take a lot of divers and a lot of diving!”) • Typical application: Once you know E[F] • I scored 3. How great am I? • If this is soccer or baseball, GREAT! • If this is basketball, time for practice…. • What is the statistical significance of a particular value of F? Lecture 18, CS567

7. Expectation - Random Sampling • Spectrum of Approaches: • Exhaustive…………………………………………………….Random Sampling • Random sampling: • Take the average of the values of the function at the random points • Examples: • “Pick a few air travel websites at random, and buy your ticket from the one giving the lowest fare” (get the average price for a market survey) • “Pick a few casinos at random, and try your luck at each” (compute the general expectation of winning at a casino) • Problems • If the number of websites/casinos is very large and/or highly variable in odds of winning (complex space), an estimate based on a small sample may be far from the global average • If a large sample is taken, begins to approximate exhaustive search • Example: Monte Carlo algorithm Lecture 18, CS567

8. Best of Both Worlds - MCMC • Pragmatic goal: To approximate expectation • “What is the average happiness in the world, give or take a laugh or two?” • “What is the average depth of the ocean, rounded off to miles?” • Pragmatic approach: • Markov Chain Monte Carlo (“Probabilistic random” sampling) • Principles: • Monte Carlo E[F(x1, x2, …xn)] = Σ{x} F(x1, x2, …xn) P(x1, x2, …xn) is approximated by E[F(x1, x2, …xn)] ~ 1/T ΣT F(tx1, tx2, …txn) Where T refers to transitions from one state of the multi-dimensional variables to another AND • Markov Chain approximation State = A particular set of values for {x} Transition to the next state depends only on current values of variables Stationary Markov chain: Constant transition probabilities Ergodic distribution: Average number of transitions between two states is equal in either direction, represented in the Markov chain transition matrix. Thus, following a series of Markov steps in the region will not alter the distribution. (Ergodicity: The distribution converges to this, irrespective of the starting distribution) Represent the distribution as a Markov chain at equilibrium (ergodic) and sample from it Lecture 18, CS567

9. MCMC – Metropolis algorithm • Goal: To sample states (and compute F for each of them) based on MCMC • Metropolis: • Separate probability of transition from state j to state i into two conditional probabilities • qik : Probability of selecting state i as the next step, while in state k (Clark Kent [while dancing with Lois Lane]: “May I have the next dance, Lana?”) • rik : Having selected state i as a candidate next step, probability of i actually becoming the next step (Lana Lang: “I’d love to, Clark! See you tomorrow, Lois”) • Thus, tik = qik rik • rik given by relative probabilities of P(si) and P(sk). If P(si) < P(sk), rik =P(si)/P(sk), else rik = 1. (If Clark usually dances with Lois Lane, then the probability of him switching partners follows the relative probabilities; if Clark usually dances with Lana, then he should switch!) • Algorithm: For x number of iterations • Start in some state sk • Pick a possible next state based on qik • Evaluate for acceptance based on rik • If accepted, calculate F(si), and back to 1 • If not accepted, back to 2 Calculate E[F(s)] Lecture 18, CS567

10. MCMC – Gibbs sampling algorithm • Goal: To sample states (and compute F for each of them) based on MCMC • Gibbs sampling: • At every step, subclassify variables into free and fixed, and evaluate probability of transition. If transition is made, the next state differs from the previous one only in the value of the free variables. • Algorithm: For x number of iterations • Start in some state Sk = (s1, s2, s3, …… sn) • Make the transition to Si based on P({sub}| sl, sl+1, …… sn-|sub|) • Repeat 2, but with a different variable freed • Choice of free variable(s) based on cycling, or probabilistic sampling Calculate E[F(s)] Lecture 18, CS567

11. Optimizing Expectation • Expectation Maximization • Example: Baum-Welch algorithm (déjà vu) • Given training data {s}, maximize E[P(s|w,M)] • “The average probability of a sequence that is part of the model should be as high as possible” or “On the average, the probability of a sequence that is part of the model should be high” • Most useful when both parameters and data are to be optimized. More generally, when two subclasses of parameters need to be optimized together. • General form (GEM) just looks for a higher/lower value, not necessarily maximum/minimum Lecture 18, CS567