Depth Enhancement via Low-rank Matrix Completion Si Lu 1 , Xiaofeng Ren 2 , and Feng Liu 1

1. Depth Enhancement via Low-rank Matrix Completion Si Lu1, Xiaofeng Ren2, and Feng Liu1 Department of Computer Science, Portland State University1Department of Computer Science and Engineering, University of Washington2 SUMMARY OBSERVATIONS: • Similar RGB-D patches approximately lie in a low-dimensional subspace. • The subspace constraint essentially captures the potentially scene-dependent image structures in the RGB-D patches in both the color and depth domain. • This low-rank subspace constraint can be enforced through incomplete matrix factorization. Depth maps captured by consumer RGB-D cameras are often noisy and miss values at some pixels. This paper presents a depth enhancement algorithm via low rank matrix completion that performs depth map completion and de-noising simultaneously. (a) Ground truth (b) Noisy RGB-D image BM3D Our method (c) Color denoising Rank distribution of 30,000 RGB-D patch matrices (a)(b) (c) (a)(b) (c) (a)(b) (c) Patch samples. (a): clean patch. (b): noisy patch. (c): the top ten eigen-vectors of the patch matrix formed by similar patches to each noisy patch. (d) Joint bilateral filter (e) BM3D + joint bilateral filter (f) Our depth enhancement result FRAMEWORK Similar patch searching Rank prediction for patch matrix Enhancement via low-rank matrix completion Training data Noisy color Color edge Features capturing patch structure properties Regression model patch matrix M matrix rank r Noisy depth Depth edge Output color Output depth Predicted rank EXPERIMENTS Input color Input depth Output color Output depth Input color Input depth Output color Output depth Input color Input depth Output color Output depth Comparisons among depth enhancement methods. This work was supported in part by NSF grants IIS-1321119, CNS-1205746, and CNS-1218589.