Stereo Vision using PatchMatch Algorithm

Stereo Vision using PatchMatch Algorithm Junkyung Kim Class of 2014

1. Updates

Updates • Source code written in C • Re-implemented PatchMatch in MATLAB • With some additional features (3)

Updates • Additional Features • 1. Ability to use weighted distance measures • Uniform by default • RBF, Gabor, Vertical Bar • (hopefully) Reduce ambiguity / being caught in local minimum

Updates • Additional Features • 2. Ability to choose sampling distribution • Exponential distribution by default • U, G, chi-squared, etc • (hopefully) accelerate convergence to optimal NNF

Updates • Additional Features • 3. Ability to further restrict search space • ‘Disparity range’ is smaller than the entire epipolar line. • (Surely) accelerate convergence & avoid false matches • User – defined parameter. Hybrid System.

2. PatchMatch on Stereo

PatchMatch on Stereo • = Constrained correspondence search algorithm • Entire image -> Epipolar line • Epipolar line -> User-defined disparity band (additional feature)

PatchMatch on Stereo • Pipeline • 1. Random Initialization • 2. Propagation • 3. Random Search • Repeat stages 2. and 3. for fixed number of iterations • Or until threshold error is reached

PatchMatch on Stereo • 1. Random Initialization • Within the search space (epipolar line by default), • randomly assign offset between two images

PatchMatch on Stereo • 1. Random Initialization • Within the search space (epipolar line by default) • randomly assign offset between two images

PatchMatch on Stereo • 2. Propagation • Compare current offset and those of previous pixels • One pixel above & One pixel to the left

PatchMatch on Stereo • 2. Propagation • Choose offset that minimizes distance measure

PatchMatch on Stereo • 3. Random Search • Sample candidate offsets in exponential distribution, centered at the current offset

PatchMatch on Stereo • 3. Random Search • Choose offset that minimizes distance measure

PatchMatch on Stereo • The algorithm is bound to reach the global minimum • As defined by global minimum of distance measure (L2 – distance by default)

PatchMatch on Stereo • Exhaustive search achieves the same solution • Why use random sampling instead of brute-force? • Expected to quickly converge the near-optimum in small number of iterations.

PatchMatch on Stereo • How quickly does this algorithm approach global minimum?

3. Results

PatchMatch on Stereo

PatchMatch on Stereo Black : Baseline / Green : Constrained disp, propagation off / Blue : Constrained disp, propagation on / Red : Exhaustive Search

4. Comments &Future Work

Comments &Future Work • Propagation is certainly helping with speed • Smoothness constraint holds • Practical advantage in constraining disparity • Stereo cameras (bumblebee, etc.) usually have fixed vergence point (+inf). • No need to consider uncrossed disparity

Comments &Future Work • PatchMatch achieves global optimum with enough iterations • Minimum of 5 with constraints • How good is the global optimum compared to ground truth? • There must be good amount of error

Comments &Future Work • Orientation disparity • Half occlusion • Distortion • Other tricks can deal with those issues better

Comments &Future Work • For project final • Try feature map for representation • Oriented edges / bars will handle orientation disparity better, more robust to distortion • Try different sampling distributions • Giving more weight at the center will increase accuracy along the physical contours

PatchMatch on Stereo • Pipeline • 1. Random Initialization • 2. Propagation • 3. Random Search • Repeat stages 2. and 3. for fixed number of iterations • Or until threshold error is reached

End

Stereo Vision using PatchMatch Algorithm