200 likes | 416 Views
Symmetric Disparity Estimation in Distributed Coding of Stereo Images. Xin Li Lane Dept. of CSEE West Virginia University Morgantown, WV 26506-6109. Outline. Background: distributed source coding (DSC)
E N D
Symmetric Disparity Estimation in Distributed Coding of Stereo Images Xin Li Lane Dept. of CSEE West Virginia University Morgantown, WV 26506-6109
Outline • Background: distributed source coding (DSC) • The gap between theory and practice: modeling nonstationary and location-related correlation structure • Distributed coding of stereo images • Asymmetric vs. symmetric approach • Side information (SI) generation: EM-like iterative disparity estimation • Coded information (CI) exploitation: jointly refine intensity and disparity uncertainty • Experimental results and summary
What is Distributed Coding? Traditional (centralized) Coding Paradigm (e.g., TV broadcasting) New (distributed) Coding Paradigm (e.g., sensor network) source Y source X source Y source X centralized encoder distributed encoder X distributed encoder Y Power-hungry component centralized decoder centralized decoder
Current State of the Art • Theoretic sources (e.g., i.i.d. binary or Gaussian) • Trellis codes: Pradhan and Ramchandran’1999, Wang and Orchard’2001 • Turbo codes: J. Garcia-Frias’2001, Aaron and Girod’2002 • LDPC codes: Stankovic, Liveris and Xiong’2002 • Practical sources (e.g., image and video) • Wyner-Ziv coding of video still significantly falls behind H.263+ (refer to “Distributed Video Coding” by Girod, Aaron, Rane and Rebollo-Monedero, Proc. of IEEE, pp.71-83 2005)
Why is Practical Source Difficult? • The gap between theory and practice: the correlation structure of practical sources is often characterized by • Nonstationary instead of stationary • Location-based instead of intensity-based • The distributed constraint of exploiting source correlation • Adaptation at the encoder is more difficult • Optimization at the decoder instead of encoder
Distributed Coding of Stereo Images Y Y X X Disparity estimation & compensation Distributed encoder X Distributed encoder Y Quantization & entropy coding Centralized decoder How to shift disparity estimation from encoder to decoder? Centralized encoder
Asymmetric vs. Symmetric Coding Left eye: X Right eye: Y Left sensor: X Right sensor: Y Amblyopia (Lazy Eye) Interlaced sampling Interlaced versions of left and right images are jointly decoded along with disparity estimation Left-eye and right-eye images with varying acuity are jointly decoded by HVS to estimate disparity. Asymmetric coding protocol Symmetric coding protocol
Symmetric Coding Protocol Primary Xeven,Yodd (finely-quantized) Secondary Xodd,Yeven (coarsely-quantized) Left image: X Right image: Y Correlation model: D: disparity compensation W: additive noise term Side Information (SI) from primary Coded Information (CI) from secondary
Side-Information (SI) Generation To estimate missing rows (SI), We need to know disparity maps To estimate disparity maps, We need to know missing rows How to solve such problem of chicken-and-egg flavor ?
EM-like Iterative Decoding Initialization (spatial interpolation) Update the estimation of missing rows (disparity-based compensation) Update the estimation of disparity maps (stereo matching/ disparity estimation)
Convergence and Caveat original reconstructed Intensity profile Energy of intensity update right left Iteration number column variable Phase misalignment (trapped in local minimum) Fast convergence of our iterative algorithm
Coded Information (CI) Exploitation Primary SI Xeven,Yodd (finely-quantized) Secondary CI Xodd,Yeven (most significant bit) Left image: X Right image: Y Kill two Birds with one Stone (secondary channel) • Work as the synchronization marker to resolve location ambiguity • Work as the quantization constraint to correct intensity errors
Benefits of Secondary Channel • Phase Alignment • Intensity Refinement Work as the synchronization marker to resolve location uncertainty CI SI original Work as the quantization constraint to correct intensity errors 201 50
Experimental Setup square tsukuba room low middle high Complexity of disparity field
Improvement of Generated SI room tsukuba Blue line: left image, green line: right image
Benefits of Secondary Channel Original EM-like scheme (no phase alignment or intensity refinement) Improved EM-like scheme (with phase alignment but no intensity refinement) Improved EM-like scheme (with both phase alignment and intensity refinement)
Disparity Map Comparison Distributed decoding Centralized encoding (The stereo matching scheme we used is fast yet less accurate)
Disparity Field Comparison Distributed decoding (uncorrected errors due to local minimum trap) Centralized encoding
Preliminary Coding Results H264 I+I Ours H264 I+P Room pair (256×256)
Summary • Distributed coding of stereo images • Why? – It calls for the modeling of nonstationary location-based correlation • How? – We propose a symmetric coding protocol in which SI is generated by EM-like iterative disparity estimation and CI is exploited to jointly refine intensity and disparity uncertainty at the decoder • What is next? – We are working on modeling more complicated correlation structure in distributed video coding