Image Segmentation

Image Segmentation • Segmentation is the process of partitioning an image into regions • region: group of connected pixels with similar properties • properties: gray levels, colors, textures, motion characteristics (motion vectors), edge continuity … • There are two approaches to segmentation • region segmentation • edge segmentation Image Segmentation

Edges and Regions • In ideal images, a region is bounded by a closed contour • the close contours may be obtained from the regions by edge detection • the regions may be obtained from the closed contours by boundary-filling • In real images, it is rare to obtain regions from contours directly and vice versa Image Segmentation

Regions and Objects • Regions are important for the interpretation of images because they may correspond to objects in a scene • An image may contain several objects and each object may contain several regions corresponding to different parts of an object • Due to a number of factors (noise, bad illumination, 3D world etc.) segmentation is usually not perfect • Image Interpretation requires object specific knowledge Image Segmentation

Image with many objects • Each object consists of several regions Image Segmentation

Region Segmentation • Region Segmentation: the pixels of the same object are grouped together and are marked to indicate that they form a region • Criteriafor region segmentation: pixels may be assigned to the same region if they have similar intensity values and they are close to one another Image Segmentation

Edge Segmentation • Find all pixels on region boundaries • Canny Operator • apply Gaussian smoothing • apply edge detection • remove false edges (e.g., noise) • thin the edge  boundary is 1 pixel wide • fill the gaps  recover missing edges • put boundary pixels in order  all pixels in a list • Ideally closed boundaries Image Segmentation

Edge Following • There may exist gaps and noisy edges in the output of Canny • Fill gaps and follow the edge in many directions • there may exist more than one regions meeting each other • not all directions are promising • continuity • proximity • intensity • length • direction Image Segmentation

Edge Following as a Graph Traversal Problem Image Segmentation

Graph Traversal • Edge following on a graph can be viewed as a minimization (or maximization) problem • the most promising path is the one which maximizes a function Φ(i,j) • promising paths correspond to strong edges • Graph search can be hard combinatorial • Can become very slow • Dynamic Programming (DP) • Heuristics Image Segmentation

Criterion Φ(C) • The edge C with maximum Φis the most promising • Φ(C) = Σφ(i,j) over all points on a curve • Σφ(i,j) = average{intensity} - average{angle} • takes averages over all points on a path • In places where the curve splits into 2 or more directions (paths), follow the direction with the maximum φ • this situation changes after a while and may be necessary to backtrack to an earlier point Image Segmentation

edges first path directions second path φ=14/3-π/(4*3) φ=17/3-0 promising at 7 promising at 6 φ=22/6-0 φ=33/6-2*π/(4*6) Image Segmentation

Using the maximum cost Φcriterion to find the cell boundaries • in microscope images • a stage in the search process • the completed boundary Image Segmentation

Region Segmentation • Analyze the grey value distribution (histogram) of the image • assumption: objects are dark against a light background • their grey-value distributions can be separated putting thresholds between them • convert a grey-level image into a binary one by applying carefully selected thresholds Image Segmentation

Many Regions • One threshold  two regions • black pixels: object • white pixels: background • More than two thresholds  more than two regions with different intensity values Image Segmentation

(1) Peakiness Detection Algorithm • Find the two highest local maxima gi, gj in the histogram that are at least at distance d apart • d is user defined • Find the lowest point gk in the histogram between gi and gj • Peakiness = min{H(gi),H(gj)}/H(gk) • Take the combination (gi,gj,gk) with the highest peakiness • Threshold the image at T=gk • N thresholds: take the N greater peakinesses (Ti=gki) Image Segmentation

(2) Iterative Algorithm • Compute a threshold T • T: mean gray value of image • Partition the image into R1, R2 using T • Compute the mean values μ1,μ2of R1,R2 • Select a new threshold T=1/2(μ1+μ2) • Repeat steps 2-4 until μ1,μ2do not change Image Segmentation

(3) Adaptive Thresholding • The same threshold is not usable everywhere in the image • e.g., uneven illumination • Partition the image into mxm subimages • Select a threshold Tij for each subimage • apply 1st or 2nd threshold selection algorithm • Compute the union of the thresholded subimages Image Segmentation

original image with uniform illumination • histogram of a • simulated uneven illumination • image a with uneven illumination added • histogram of image d • thresholding at T=72failed!! Image Segmentation

(4) Double Thresholding • Select threshold T1 (apply 1st or 2nd algorithm) • Set T2 = T1 + ε (εis user defined) • Compute regions R1, R2, R3: • R1: pixels with gray values less than T1 • R2: pixels with gray values between T1 and T2 • R3: pixels with gray values above T2 • Visit each pixel of R2: if the pixel has a neighbor in R1, reassign it to R1 • Repeat step 4 until no pixels of R2 are reassigned • Reassign all pixels remaining in R2 to R3 Image Segmentation

Image Segmentation