1 / 49

Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching

Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching. Cuong Cao Pham and Jae Wook Jeon , Member, IEEE. IEEE Transactions on Circuits and Systems for Video Technology, 2012. Outline. Introduction Framework Proposed Algorithm Compute Costs

gaenor
Download Presentation

Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching Cuong Cao Pham and Jae WookJeon, Member, IEEE IEEE Transactions on Circuits and Systems for Video Technology, 2012

  2. Outline • Introduction • Framework • Proposed Algorithm • Compute Costs • Cost Aggregation : Domain Tramsformation • Optimization & Refinment • Experimental Results • Conclusion

  3. Introduction

  4. [4] K.-J. Yoon and I.-S. Kweon, “Adaptive Support-Weight Approach for Correspondence Search,” IEEE Trans. Pattern Anal. Mach. Intell., vol.28, no. 4, pp. 650-656, 2006. Background • Global stereo algorithms: • High accuracy but low speed • Local stereo algorithms : • High speed but low accuracy • The key : cost aggregation • Adaptive support-weight[4] : • ‧The most well-known local method • ‧The state-of-art local algorithm • ‧Reduce the gap between global method and local method → Excessive time consumption related to support window size

  5. Related Work • Adaptive Weight[4] • Bilateral filter • Cost-volume filtering[21] • Guided filter • Geodesic Diffusion[27] • Anisotropic diffusion → Geodesicdiffusion [21] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz, “Fast Cost-Volume Filtering for Visual Correspondence and Beyond,” in Proc.IEEEIntl. Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 3017-3024,2011. [27] L. De-Maeztu, A. Villanueva, and R. Cabeza, “Near Real-Time Stereo Matching Using Geodesic Diffusion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 2, pp. 410 - 416, 2012.

  6. Objective • Present a cost aggregation technique: • Achieve high precision • Fast execution • Using Domain transformation • Domain transformation: • Aggregation of 2D cost data → a sequence of 1D filters • Lower computational requirements

  7. Framework

  8. Framework

  9. ProposedAlgorithm

  10. Pixel-wise Cost Consumption • Truncated absolute difference (TAD) : • TAD of the gradient : • Final cost data: Ii(p):intensity value of the i-th color channel in the RGB color space at pixel p of the image I Tc: user-defined truncation value

  11. Aggregation 1D Cost Data • Inspired by the domain transformation technique[14] • Dimensionality reduction technique • Defines a geodesic distance-preserving representation of a 2D image embedded in 5D (x, y, Ir, Ig, Ib) as a real line. • Aggregation of 2D cost data → a sequence of 1D filters • Reduce computational time [14] Eduardo S. L. Gastal and Manuel M. Oliveira, “Domain Transform for Edge-Aware Image and Video Processing,” ACM Trans. Graph., vol. 30, no. 4, 2011.

  12. Aggregation 1D Cost Data • 1D discrete signal: • Cost slide Cd : • Feedback comb filter[32]: • Cd,y : input signal • Cd,y : output signal • a feedback coefficient row y a : consistent → non-edge-aware filter ‘ n-1 n [32] J. Smith, “Introduction to Digital Filters with Audio Applications,” W3K Publishing, 2007.

  13. Aggregation 1D Cost Data • 1D discrete signal: • Cost slide Cd : • Feedback comb filter[32]: • Cd,y : input signal • Cd,y : output signal • a feedback coefficient row y a : consistent → non-edge-aware filter ‘ n-1 n

  14. Aggregation 1D Cost Data • Two similar samples set a high value of a • Two different samples set a low value of a (Discontinue region →prevent the propagation train) • Edge-aware feedback comb filter: • g: chosen metric representing the dissimilarity between two samples • Compute g as the distance between two samples in the 1D domain (transformed from the corresponding row of the guidance image I)

  15. Domain Transformation • I:R2→R3(a2D RGB color image) • p = (xp,yp): spatial coordinate • I(p) = (rp, gp, bp) :range coordinates • Goal: find a transform t :R2→R which preserves the original distances between points on C (given by some metric) R2 R3 g v

  16. Domain transformation • L1 distance between two neighboring points in the original domain R2 • Distance between two corresponding samples in the new domain R • gt(x) = t (x, I(x)) : the transformation operator at point x must equal R R2

  17. Domain transformation • Divide both sides by h and take the limit as h→0: • The value at any point u in the transformed domain: (By taking the integral of gt′ (x) from 0 to u)

  18. Domain transformation • The value at any point u in the transformed domain: • The distance between any two points u and v in the transformed domain : (corresponds to the arc length from u to v of the signal I)

  19. Domain transformation • The distance between any two points u and v : • We can also control the influence of spatial and intensity range information similar to the bilateral filter. • Embedding the values of σsand σr:

  20. Domain transformation • Select the maximum absolute difference to define the distance between two points in the original domain: • The final distance g:

  21. Domain transformation Left image Non-edge-aware filter Edge-aware filter

  22. Aggregation 2D Cost Data

  23. Aggregation 2D Cost Data • 1. Left → Right • 2. Right→Left • 3. Top → Bottom • 4. Bottom→ Top

  24. Aggregation 2D Cost Data L→ R R→ L T→ B B→ T

  25. Aggregation 2D Cost Data • is the 1D discrete signal plotted from each column along the y direction of the cost slide Cd • : • σH: kernel standard deviation (implicitly set to σs) • σs[10,300] andσr[0.01,0.3] can yields good results.

  26. Aggregation 2D Cost Data ‧Algorithm:

  27. Optimization & Refinement • Winner-take-all • Select disparities • Left-Right consistency check • Occluded regions • Weighted median filter • Noise removing

  28. Winner-take-all • Winner-take-all(WTA) strategy: • Sd: the set of all possible disparities • Cd: Aggregated cost ‘

  29. Left-right consistency check • The disparity maps obtained at this stage contain errors in the occluded regions. • Perform Left-right consistency check • A pixel in the left disparity map is marked as invalidated: when its value differs from the corresponding value of the pixel in the right disparity map by a value greater than one • Assign the minimum value between two closest validated pixels min validated Left image Right image invalidated

  30. Weighted Median Filter • Using a weighted median filter to : • Remove streak-like artifacts • Remove the small amount of remaining noise • Select bilateral filter weight to compute the weighted median filter • The validated pixels are not affected by this operation.

  31. Consistency Map vs. Final disparity Invalidated pixels

  32. ExperimentalResults

  33. Experimental Results • Middlebury stereo evaluation • Middlebury dataset • Real-world image • Camcorder data • Execution time • CUDA implementation

  34. Middlebury Evaluation - 1 • Adaptive Weight[4] • 3535 support window with γs= 17 and γr= 7:5 • Cost-volume filtering[21] • 19×19 support window and ε = 0:0004 • Geodesic Diffusion[27] • Iterated n = 24 times with γc = 40 and l0= 0:15 • InfoPermeable[31] • Exponential function with σ = 25 • Proposed • σs=25 and σr=0.1 Compare with the best-performing algorithm inspired by well-known edge-aware filters [31] C. Cigla and A. A. Alatan, “Efficient Edge-Preserving Stereo Matching”,in ICCV Workshop on LDRMV, 2011.

  35. Middlebury Evaluation - 1 • Compare the performance of the raw cost aggregation • The same pixel-wise cost computation and disparity optimization steps were installed to ensure fair comparison. • Select the TAD of the color and the gradient for computing matching costs • { λ , Tc, Tg}={ 0.1, 7/255,2/255 } • Guidance image used for the aggregation stage: • Using 3x3 median filter • Reduce the high-frequncy information that is not actually useful

  36. Experimental Results Onlynon-occluded and discontinuity regions

  37. Middlebury Evaluation - 2 • Without refinement vs. with refinement • { λ , Tc, Tg, σs , σr }={ 0.1, 7/255,2/255, 45, 0.006 } • 3x3 median filter • Filtering Guidance image used for the aggregation stage • The weighted median filter • Used in disparity refinement stage • r = 21, γs= 81, and γr= 0.04

  38. Experimental Results with refinement without refinement

  39. Experimental Results

  40. Experimental Results

  41. Experimental Results

  42. Real-world Image • Camcorder data: • Cafe (640360, 32 possible disparities) • Newspaper (512384, 32 possible disparities) • Book_Arrival (512384, 60 possible disparities)

  43. Proposed vs. CostFilter[21]

  44. Execution time • Using C++ • PC with an AMD Athlon 64 X2 Dual Core 3800+ 2.00 Ghz. • Measure only the execution time of the aggregation performing on the left view • No occlusion handling or post-processing times were included.

  45. Execution time Iteration times: n Window: 2n+1 2n+1 Window: 2n+1 2n+1 Support window size / number of iterations

  46. CUDA Implementation

  47. Conclusion

  48. Conclusion • Solve the excessive time consumption bottleneck of adaptive-weight • Integrates the appealing properties of domain transformation into the cost aggregation • Using a sequence of 1D operations • Lower computational requirements • Lower memory costs • Fast and accurate local method

More Related