380 likes | 554 Views
Advances in Adaptive Weight Support Windows. Barry McCullagh, Keimyung University Daegu. Presentation Overview. Advances in adaptive support weight windows: Introduce the problems of fixed support weights. Seminal paper by Yoon and Kweon. Advances in the past 12-18 months.
E N D
Advances in Adaptive Weight Support Windows Barry McCullagh, Keimyung University Daegu
Presentation Overview • Advances in adaptive support weight windows: • Introduce the problems of fixed support weights. • Seminal paper by Yoon and Kweon. • Advances in the past 12-18 months. • Future research. • Questions and answers.
Correlation in stereovision • Correlation windows (support windows) are used to locate matching pixels in stereo image pairs.
Window comparison • Traditionally each pixel was given the same weight. • Comparison performed using SAD, SSD, NCC or other metric. • Different objects in a window?
Adaptive weights • First introduced by Yoon and Kweon from KAIST. • Importance of pixel depends on: • Color similarity to center pixel. • Distance to the center pixel.
Color similarity • Pixels in the blue boundary are more important than the other pixels because they are of similar color.
Since Yoon and Kweon • Use of adaptive weights has become popular: • Geodesic support weights • Biologically inspired weighting • Disparity Calibration Systems
Geodesic Support Weights • Local Stereo Matching Using Geodesic Support Weights. • Honsi, Bleyer, Gelautz, Rhemann, ICIP 2009. • Looks at the path to the center pixel. • Support weight is high if the pixels on the path are of similar color. • Support weight is low if the pixels on the path are dissimilar.
Path to the center • Problem identified in this paper: • Similar colors separated by dissimilar colors might not belong to the same object. • Not at the same depth.
Original paper: some areas are given incorrectly high levels of support.
Geodesic Technique • Geodesic technique looks at the path from a pixel in the window to the center pixel. • Consistent colors indicate the same object and therefore the same disparity.
Results • Image • Original Weighting • Geodesic Weighting
Biologically Inspired Weighting • “Biologically and psychophysically inspired adaptive support weights algorithm for stereo correspondence” Lazaros Nalpantidis and Antonios Gasteratos • Desire to simulate human matching in computers. • Identify techniques humans use and model these in stereo matching programs.
Biological Matching • Color Matching • Circular Windows • Gestalt Laws • Psychophysically-based weight assignment. • All expanding on the Absolute Difference calculated between individual pixels at different disparities.
Color Matching – color space • Color images produce more accurate results than color. • Different color spaces produce different levels of accuracy: • CIELab better than RGB • However: • color sensors are RGB. • RGB computations are less demanding.
Color Matching – weighting factors • Human Visual System Weightings: • 0.299 Red • 0.587 Green • 0.114 Blue • Stereovision: Equal weightings • Equal amount of information.
Circular Windows • Most approaches use rectangular windows. • Biological model is better approximated using circular windows. • Eye is circular • Contribution of neighboring pixels is perfectly isotropic.
Gestalt Laws • Gestalt – relationships that bond single items to make a group. • Makes a pattern instead of parts. • Pattern has different characteristics than the parts. • These laws are useful to help locate matching objects in stereovision.
Gestalt Laws • Elements (pixels) making up a group (belong to a larger object) are governed by the following rules: • Proximity: elements that are close to each other. • Similarity: elements similar in an attribute (color, etc). • Continuity: elements that could belong to a smooth larger feature. • Common fate: elements that exhibit similar behavior. • Closure: elements that could provide closed curves. • Parallelism: elements that seem to be parallel. • Symmetry: elements that exhibit a larger symmetry.
Gestalt Laws for image processing • Three most basic laws can be used to assist matching: • Proximity (or equivalently Distance). • Intensity similarity (or equivalently Intensity dissimilarity). • Continuity (or equivalently discontinuity).
Psychophysically-based weight assignment • Assign support weights based on the human response. • Weber-Fechner law shows the relationship between perceived change in stimulus and actual change in stimulus
Combining these laws • Three basic Gestalt laws are combined: • Wtotal = Wdist * Wdisc * Wdissim
Performance • Less accurate than the original algorithm from Yoon and Kweon. • Provides a basis for modeling the human visual system in computing.
Expanding on Adaptive Weights • “Local stereo matching with adaptive support-weight, rank transform and disparity calibration” Zheng Gu, Xianyu Su, Yuankun Liu and Qican Zhang. • Uses adaptive weight as the starting point and applies additional techniques to the result.
Additional Steps • Rank Transform • Pixels assigned discrete weights based on similarity to center pixel. • Disparity Calibration • Selection of window • Disparity calculation
Rank Transform • Calculates the intensity differences between the center pixel and pixels in the support windows. • If corresponding pixels in the two support windows have different differences, then those pixels are not included
Disparity Calibration • Selection of window: applies adaptive weights but with stronger emphasis on color similarity. • Recalculation of disparity values based on those of surrounding pixels
Disparity recalculation • Examine the disparity values for all pixels in the support window. • Assign the most common disparity to that of the center pixel. • Removes outliers • Increase smoothness of the disparity map.
Performance • Performs better than the original adaptive weight and better than most local methods.
Future Research • Two main goals: • improve speed by • re-implementing algorithms on parallel architectures. • modifying algorithms to reduce the computational cost. • improve accuracy: • using more complex algorithms. • merging components of different algorithms.
Parallel Architectures • GPUs, CBE, Multi-core CPU • Powerful, ubiquitous and cost effective. • Efficient implementation of adaptive support weight algorithms will achieve real-time rates. • Adaptive Support Weight • implementable on parallel devices. • may be restricted by local memory, available instructions and order of execution.
Modifying Algorithms • Computation cost is proportional to window size: • sliding windows cannot be used in most cases. • ASW often use windows of 35x35 pixels or larger. • Investigate techniques to reduce the window size while maintaining accuracy: • hierarchical approaches. • adaptively sized windows.
Modifying Algorithms • Combining and modeling more of the Gestalt laws. • Varying the importance of these laws. • Geodesic supports show color is very important. • Are other laws more important?
Merging Algorithms • Adaptive Support Weights forms the basis of many algorithms. • e.g. ASW combined with rank transform, modified to allow use with curvelets. • What other techniques can ASW be combined with? • As an initial step. • As a final step.