370 likes | 444 Views
An Investigation Into Segmenting Traffic Images Using Various Types of Graph Cuts. Jonathan Dinger. Analysis of Traffic Videos Using Computer Vision. Traffic footage example. Segmentation. Important step in video analysis Background subtraction is often used. Background Subtraction.
E N D
An Investigation Into Segmenting Traffic Images Using Various Types of Graph Cuts Jonathan Dinger
Analysis of Traffic Videos Using Computer Vision Traffic footage example
Segmentation • Important step in video analysis • Background subtraction is often used
Background Subtraction • Uses pixelwise computations • Performance could be better • Better segmentation = better traffic detection
Graph Cuts • Use interaction between neighboring pixels • Keep objects segmented together • Better segmentations
Graphs • G = (V,E) • Two vertices in V called source and sink • s and t, respectively • Remaining vertices called M • Vertices in M connected to both s and t (T-links) • Vertices in M connected to neighboring vertices (N-links)
Graphs • T-links are uni-directional • N-links are bi-directional • Each edge has a weight • also known as a capacity • Each pixel has its own vertex
Definitions • Path • List of vertices connected by edges • s-t cut • Removal of edges such that all vertices have a path to either the source or the sink, but not both • Flow • Each edge has a capacity • That much flow can be pushed through each edge • Flow through a graph is the cumulative amount of flow going from the source to the sink
Definitions • Minimum cut/maximum flow • A cut with the smallest cost (weight) • Maximum flow that can be pushed from source to sink (capacity) • By max-flow min-cut theorem, these are equal • Graph cuts • Minimum cut on a graph
Augmenting Paths Algorithm • Method to find maximum flow • Augmenting path • Path where flow can be increased in all edges between vertices in path • Run search • Find augmenting path from source to sink • Add more flow to that path • Loop back to 1.
Augmenting Paths Example Step 1. Step 2. Step 3. Step 4.
Augmenting Paths Example Step 5. Step 6. Step 7. Step 8.
Implementations • Two separate implementations • Our implementation • Kolmogorov’s implementation http://www.cs.ucl.ac.uk/staff/V.Kolmogorov/software/maxflow-v3.01.src.tar.gz
Banded Graph Cuts (BGC) • Downsampled images • Cut performed over smaller area • Then upsample, and perform cut over band • Faster than graph cuts • Less detailed than graph cuts
BGC with Laplacian Augmentation • Downsampling images loses information • Use Laplacian pyramid to store lost data • Computes difference between image and image gained by downsampling and upsampling again • Add back some of the lost detail • Resegment in areas where detail was lost
GC vs BGC • BGC faster, less accurate • GC slightly slower, more accurate Graph cut Banded graph cut
Multi-way Graph Cut • 3 or more labels • α-β swap • Loop through label pairs • Run graph cut on current pair of labels • If current label of a pixel is not one of the pair, do not use pixel in graph cut • Graph cuts will swap some pixels with label α to label β and vice versa
Traffic Segmentation • Graph cut on each image • Background image computed per pixel • N is the number of images, xi is the grayscale value of the current pixel, and μ is the average grayscale value over all image frames
Traffic Segmentation • Find variance of each pixel • N, xi, and μ are as above • σ2 is the variance • Threshold the variance • If variance is below the threshold, do not include pixel in graph cut • Assume non-varying pixels are background pixels • Avoid divide-by-zero errors in weights
T-link Weights • Exponentials • x is the grayscale value of the current pixel • μ and σ2 are as above • β is a constant that forces the two functions to be equal at α standard deviations
T-link Weights • Absolute differences • x and μ are as above • L is the maximum possible distance between x and μ, so for grayscale images • K is a shift constant that forces f3 and f4 to be equal at α standard deviations
More T-link Weights • Simple Grayscale Differences • x is the grayscale value of the pixel. Cm and Cn are two grayscale values used as a basis for segmentation
N-link Weights • Distance • Euclidean distance between two neighboring pixels with coordinates (x1, y1) and (x2, y2) • Smoothing (similarity) • x and y are the grayscale values of two neighboring pixels. • is the maximum possible difference between the pixel values • γ is a modifier that defines the amount of smoothing that takes place
Results for Graph Cuts Image Graph cut Banded graph cut Augmented BGC Our graph cut Multi-way cut
Timing Results Computing times for cut results in milliseconds (ms)
Results for Traffic Segmentation (distance) Image Background Variance Exponential cut Absolute difference cut
More results (distance) Image Exponential Absolute difference
More results (distance) Image Exponential Absolute difference
Comparison of Distance Model with Background Subtraction Image Exponential Background subtraction
Results (smoothing) Image Exponential Absolute difference
More results (smoothing) Image Exponential Absolute difference
Comparison of Smoothing Model with Background Subtraction Image Absolute difference Background subtraction
Video Results (smoothing) Image Graph cut Background Subtraction
Conclusions • Segmentation performance without smoothing comparable to background subtraction • Background subtraction is faster, easier • Smoothing model • Segments larger pieces of vehicles into one section • Vehicle segmentations more “solid” • Absolute difference T-link weights combined with smoothing N-link weights give best results
Future Work • Use multi-way cuts to add shadow segmentation • Extend to RGB