720 likes | 1.09k Views
EM for Motion Segmentation. By: Yair Weiss and Edward H. Adelson. Presenting: Ady Ecker and Max Chvalevsky. .
E N D
EM for Motion Segmentation By: Yair Weiss and Edward H. Adelson.Presenting: Ady Ecker and Max Chvalevsky. “Perceptually organized EM: A framework that combines information about form and motion”“A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models”
Contents • Motion segmentation. • Expectation Maximization. • EM for motion segmentation. • EM modifications for motion segmentation. • Summery.
Part 1: MotionSegmentation
Motion segmentation problem v • Input: 1. Sequence of images.2. Flow vector field – output of standard algorithm. • Problem: Find a small number of moving objects in the sequence of images. vy vx
flow data Segmentation Output • Classification of each pixel in each image to its object. • Full velocity field. velocity field
Motion vs. static segmentation • Combination of motion and spatial data.Object can contain parts with different static parameters (several colors). • Object representation in an image can benon-continuous when: • There are occlusions. • Only parts of the object are captured...
Difficulties • Motion estimation. • Integration versus segmentation dilemma. • Smoothing inside the model while keeping models independent.
Motion estimation - review • Estimation cannot be done from local measurements only. We have to integrate them.
Motion integration • In reality we will not have clear distinction between corners and lines.
Integration without segmentation • When there are several motions, we might get false intersection points of velocity constraints at T-junctions.
Integration without segmentation • False corners (T-junctions) introduce false dominant directions (upwards).
Contour ownership • Most pixels inside the object don’t supply movement information. They move with the whole object.
Smoothing • We would like to smooth information inside objects, not between objects.
Human segmentation • Humans perform segmentation effortlessly. • Segmentation may be illusive. • Tendency to prefer (and tradeoff): • Small number of models. • Slow and smooth motion. • The segmentation depends on factors such as contrast and speed, that effect our confidence in possible motions.
Part 2: ExpectationMaximization
Clustering Problems • Structure: • Vectors in high-dimension space belong to (disjoint) groups (clusters, classes, populations). • Given a vector, find its group (label). • Examples: • Medical diagnosis. • Vector Quantization. • Motion Segmentation.
Start with random model parameters Maximization step: Find the center (mean)of each class Expectation step: Classify each vectorto the closest center EM: Unknown clusters and centers
EM Characteristics • Simple to program. • Separates the iterative stage to two independent simple stages. • Convergence is guaranteed, to some local minimum. • Speed and quality depend on: • Number of clusters. • Geometric Shape of the real clusters. • Initial clustering.
Soft EM Each point is given a probability (weight) to belong to each class. • The E step:The probabilities of each point are updated according to the distances to the centers. • The M step:Class centers are computed as a weighted average over all data points.
Soft EM (cont.) • Final E step:classify each point to the nearest (most probable) center. • As a result: • Points near a center of a cluster have high influence on the location of the center. • Points near clusters boundaries have small influence on several centers. • Convergence to local minima is avoided aseach point can softly change its group.
Perceptual Organization • Neighboring or similar pointsare likely to be of the same class. • Account for this in the computation of weights by prior probabilities.
Example: Fitting 2 lines to data points (xi,yi) • Input: • Data points that where generated by 2 lines with Gaussian noise. • Output: • The parameters ofthe 2 lines. • The assignment of each point to its line. ri y=a1x+b1+sv y=a2x+b2+sv v~N(0,1)
The E Step • Compute residuals assuming known lines: • Compute soft assignments:
Least-Squares review • In case of single line and normal i.i.d. errors, maximum likelihood estimation reduces to least-squares: • The line parameters (a,b) are solutions to the system:
The M Step • In the weighted case we find • Weighted least squares system is solved twice for (a1,b1) and (a2,b2).
Estimating the number of models • In weighted scenario, additional models will not necessarily reduce the total error. • The optimal number of models is a function of the s parameter – how well we expect the model to fit the data. • Algorithm: start with many models. redundant models will collapse.
Illustration l=log(likelihood)
Part 3: EM for MotionSegmentation
Segmentation of image motion: Input Products of image sequence: • Local flow – output of standard algorithm. • Pixel intensities and color. • Pixel coordinates. • Static segmentation: • Based on the same local data. • Problematic as explained before.
Segmentation output • segmentation • Models: • ‘blue’ model ‘red’ model
Notations • r - pixel. • Or - flow vector at pixel r. • k - model id. • qk - parameters of model k. • vk(r) - velocity predicted by model k at location r. • Dk(r) = D(r, qk) - distance measure. • s - expected noise variance. • gk(r) - probability that pixel ‘r’ is a member of model ‘k’.
Segmentation output • Segmented O: • Model parameters: • bluered r Vblue(r) Vred(r) O(r)
The E Step • Purpose: determine statistic classification of every pixel to models. • pk(r) - prior probability granted to model ‘k’. • For classical EM, pk(r) are equal for all ‘k’.
The E Step (cont) • Alternative representation: Soft decision enables slow convergence to better minimum instead of finding local minima.
Distance measure functionality • Correct physical interpretation of motion data. • If possible – enable analytic solution.
Distance measures (1) • Optic flow constraint: • a – window centered at ‘r’. • vk(r) – velocity of ‘k’ at location ‘r’. • Quadratic. Provides closed MLE solution for the M-step.
Distance measures (2) • Deviation from constant intensity: • a – window centered at ‘r’. • Good for high speed motion. • Resolved by successive linearizations.
The M step • Purpose: layer optimization(according the soft classification of pixels). • Produces weighted ‘average’ of the model. • ‘Average’ depends on definition of D. • Constrained by J (slow & smooth motion).
J (cost) definition • For loosely constrained q(typical for image segmentation): • For highly constrained q:(#degrees of freedom < #owned pixels). • l® 0
Start with random model parameters Maximization step: Find the center (mean)of each class Estimation step: Classify each vectorto the closest center EM: Unknown clusters and centers