Gaussian Mixture Models and Expectation-Maximization Algorithm

1. Gaussian Mixture Models andExpectation-Maximization Algorithm

2. The RGB Domain • A regular image

3. The RGB Domain • Image pixels in RGB space

4. Pixel Clusters • Suppose we cluster the points for 2 clusters

5. Pixel Clusters • The result in image space

6. Normal Distribution (1D Gaussian)

7. 2D Gaussians • d = 2 • x = random data point (2D vector) • = mean value (2D vector) • = covariance matrix (2D matrix) • The same equation holds for a 3D Gaussian

8. 2D Gaussians

9. Exploring Covariance Matrix • is symmetric • has eigendecomposition (svd)

10. Covariance Matrix Geometry b a

11. 3D Gaussians

12. GMMs – Gaussian Mixture Models W H • Suppose we have 1000 data points in 2D space (w,h)

13. GMMs – Gaussian Mixture Models W H • Assume each data point is normally distributed • Obviously, there are 5 sets of underlying gaussians

14. The GMM assumption • There are K components (Gaussians) • Each k is specified with three parameters: weight, mean, covariance matrix • The total density function is:

15. The EM algorithm (Dempster, Laird and Rubin, 1977) Raw data Total Density Function GMMs (K = 6)

16. EM Basics • Objective: Given N data points, find maximum likelihood estimation of : • Algorithm: 1. Guess initial 2. Perform E step (expectation) • Based on , associate each data point with specific gaussian 3. Perform M step (maximization) • Based on data points clustering, maximize 4. Repeat 2-3 until convergence (~tens iterations)

17. EM Details • E-Step (estimate probability that point t associated to gaussian j): • M-Step (estimate new parameters):

18. EM Example data point t blue: wt,j Gaussian j

19. EM Example

20. EM Example

21. EM Example

22. EM Example

23. EM Example

24. EM Example

25. EM Example

26. Back to Clustering • We want to label “close” pixels with the same label • Proposed metric: label pixels from the same gaussian with same label • Label according to max probability: • Number of labels = K

27. Graph-Cut Optimization

28. Motivation for Graph-Cuts • Let’s recall the car example

29. Motivation for Graph-Cuts • Suppose we have two clusters in color-space • Each pixel is colored by it’s associated gaussian index

30. Motivation for Graph-Cuts • A Problem: Noise • Why? Pixel labeling is done independently for each pixel, ignoring the spatial relationships between pixels!

31. Formalizing a New Labeling Problem • Previous model for labeling: • A new model for labeling. Minimize E: • f = Labeling function, assigns label fp for each pixel p • Edata = Data Term • Esmooth = Smooth Term • Lamda is a free parameter

32. The Energy Function • Labels Set: { j=1,…,K } • Edata: • Penalize disagreement between pixel and the GMM • Esmooth: • Penalize disagreement between two pixels, unless it’s a natural edge in the image • dist(p,q) = normalized color-distance between p,q

33. Minimizing the Energy • Solving Min(E) is NP-hard • It is possible to approximate the solution using iterative methods • Graph-Cuts based methods approximate the global solution (up to constant factor) in polynomial time • Read: “Fast Approximate Energy Minimization via Graph Cuts”, Y. Boykov, O. Veksler and R. Zabih, PAMI 2001

34. α-expansion moves • When using iterative methods, each iteration some of the pixels change their labeling • Given a label α, a move from partition P (labeling f) to a new partition P’ (labeling f’) is called an α-expansion move if: Current Labeling One Pixel Move α-β-swapMove α-expansionMove

35. Algorithm for Minimizing E(f) 1. Start with an arbitrary labeling 2. Set success = 0 3. For each label j 3.1 Find f’ = argmin(E(f’)) among f’ within one α-expansion of f 3.2 If E(f’) < E(f), set f = f’ and success = 1 4. If (success == 1) Goto 2 5. Return f How to find argmin(E(f’)) ?

36. A Reminder: min-cut / max-flow • Given two terminal nodes α and β in G=(V,E), a cut is a set of edges C E that separates α from β in G’=(V,E\C) • Also, no proper subset of C separates α from β in G’. • The cost of a cut is defined as the sum of all the edge weights in the cut. The minimum-cut of G is the cut C with the lowest cost. • The minimum-cut problem is solvable in practically linear time.

37. Finding the Optimal Expansion Move • Problem: Find f’ = argmin(E(f’)) among f’ within one α-expansion of f • Solution: Translate the problem to a min-cut problem on an appropriately defined graph.

38. Graph Structure for Optimal Expansion Move Terminal α • 1-1 correspondence between cut and labeling • E(f) is minimized! Cut C Terminal not(α)

39. A Closer Look • Each pixel gets a node P1 P2 Pα

40. A Closer Look • Add auxiliary nodes between pixel with different labels P1 P2 Pα

41. A Closer Look • Add two terminal nodes for α and not(α) P1 P2 Pα

42. A Closer Look P1 P2 Pα

43. A Closer Look P1 P2 Pα

44. A Closer Look P1 P2 Pα

45. A Closer Look P1 P2 Pα

46. A Closer Look P1 P2 Pα

47. Implementation Notes • Neighboring system can be 4-connected pixels,8-connected and even more. • Lamda allows to determine the ratio between the data term and the smooth term. • Solving Min(E) is simpler and possible in polynomial time if only two labels involved(see “Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images”, Y. Boykov and M-P. Jolly 2001) • There is a ready-to-use package for solving max-flow(see http://www.cs.cornell.edu/People/vnk/software/maxflow-v2.2.src.tar.gz)

48. Final ProjectOptimized Color Transfer www.cs.tau.ac.il/~gamliela/color_transfer_project/color_transfer_project.htm