3-D Computer Vision CSc 83020

1. 3-D Computer Vision CSc 83020 Clustering methods and boundary representations 3-D Computer Vision CSc 83020 – Ioannis Stamos

2. Image Segmentation • Generate clusters (regions) of pixels that correspond to meaningful entities. • Use metrics of “closeness” between values. • Use algorithms for combining “close” values. • Apply other constraints (connectivity). 3-D Computer Vision CSc 83020 – Ioannis Stamos

3. Example (Forsyth & Ponce) 3-D Computer Vision CSc 83020 – Ioannis Stamos

4. Simple Clustering Methods • Divisive clustering • Everything is a big cluster at beginning • Split recursively • Agglomerative clustering • Each data (pixel) is a cluster • Merge 3-D Computer Vision CSc 83020 – Ioannis Stamos

5. Simple clustering 3-D Computer Vision CSc 83020 – Ioannis Stamos

6. Simple clustering • What is a good inter-cluster distance? • How many clusters are there? 3-D Computer Vision CSc 83020 – Ioannis Stamos

7. Simple clustering 3-D Computer Vision CSc 83020 – Ioannis Stamos

8. Clustering by K-means • Assume that the number of clusters (k) is known. • Each cluster has a center Ci (i=1..k) • Each data-point is a vector xj (j=1..Number of pixels) • Examples: xj=[x-coord, y-coord, gray-value] or xj=[gray-value] or xj=[red-value, green-value, blue-value] • Assume that elements are close to center of clusters. • Minimize: 3-D Computer Vision CSc 83020 – Ioannis Stamos

9. Clustering by K-means • Iterative algorithm: • Allocate each point to center of closest cluster (assuming centers are known) • Calculate centers of clusters (assuming allocations are known) • How do we start? 3-D Computer Vision CSc 83020 – Ioannis Stamos

10. Clustering by K-means 3-D Computer Vision CSc 83020 – Ioannis Stamos

11. Example (Forsyth & Ponce) 3-D Computer Vision CSc 83020 – Ioannis Stamos

12. Example (Forsyth & Ponce) 3-D Computer Vision CSc 83020 – Ioannis Stamos

13. Example (Forsyth & Ponce) 3-D Computer Vision CSc 83020 – Ioannis Stamos

14. RANSAC • RANdom SAmple Consensus • Model fitting method • Line-fitting example • Fitting a line to a set of edges with 50% outliers • Least squares would fail • Solution: M-estimator or RANSAC 3-D Computer Vision CSc 83020 – Ioannis Stamos

15. RANSAC (line-fitting example) • Two edges (wout normal) define a line. • General idea: • Pick two points. • Write the equation of the line. • Check how many other points are “close” to line. • If number of “close” points is above threshold, done • Otherwise, pick two new points. • Questions: • Which points to pick? • Complexity in worst case? 3-D Computer Vision CSc 83020 – Ioannis Stamos

16. RANSAC (line-fitting example) 3-D Computer Vision CSc 83020 – Ioannis Stamos

17. RANSAC (line-fitting example) How large should k (max. number of iterations) be? 3-D Computer Vision CSc 83020 – Ioannis Stamos

18. RANSAC (line-fitting example) How large should k (max. number of iterations) be? • Assume that w is the probability of picking a “correct” point (i.e. a point on the line). • Since we are picking n (=2 for lines) points, 1-wn is the probability of picking n “wrong” points. • If we iterate k times we want the probability of failure to be small: i.e. (1-wn)k = z => k = log(z)/log(1-wn) • If z=0.1 and w=0.5 then k=8 (n=2) • If z=0.01 and w=0.5 then k=16 (n=2) • If z=0.001 and w=0.1 then k = 687 (n=2) • How is the formula affected by n? 3-D Computer Vision CSc 83020 – Ioannis Stamos

19. RANSAC (Conclusions) • When can this method be successful? • Can we detect circles? • In that case how many points do you need to fit a circle? • Can we detect other shapes? 3-D Computer Vision CSc 83020 – Ioannis Stamos

20. Boundary representation of regions 3-D Computer Vision CSc 83020 – Ioannis Stamos

21. Representation of 2-D Geometric Structures • To MATCH image boundary/region with MODEL • boundary/region, they must represented in the same • manner. • Boundary Representation • Snakes – Extraction of arbitrary contours from image. • Region Representation 3-D Computer Vision CSc 83020 – Ioannis Stamos

22. Representation Issues • Compact: Easy to Store & Match. • Easy to manipulate & compute properties. • Captures Object/Model shape. • Computationally efficient. 3-D Computer Vision CSc 83020 – Ioannis Stamos

23. Boundary Representation Polylines: concatenation of line segments. Breakpoint Matching on the basis of: # of line segments lengths of line segments angle between consecutive segments 3-D Computer Vision CSc 83020 – Ioannis Stamos

24. Running Least Squares Method ei Move along the boundary At each point find line that fits previous points (Least Squares) Compute the fit error E=Sum(ei) using previous points If E exceeds threshold, declare breakpoint and start a new running line fit. 3-D Computer Vision CSc 83020 – Ioannis Stamos

25. Approximating Curves with Polylines • Draw Straight line between end-points of curve • For every curve point find distance to line. • If distance is less than tolerance level for all points, Exit • Else, pick point that is farthest away and use as breakpoint. • Introduce new segments. • Recursively apply algorithm to new segments. 3-D Computer Vision CSc 83020 – Ioannis Stamos

26. Ψ-s Curve y Ψ 2π 6 5 π 4 3 ψ s 1 2 x s y 4 5 3 6 1 2 x 3-D Computer Vision CSc 83020 – Ioannis Stamos

27. Ψ-s Curve y Ψ 2π 6 5 π 4 3 ψ s 1 2 x s y 4 Ψ-s is periodic (2π) Horizontal section in Ψ-s curve => straight line in the Image Non horizontal section in Ψ-s => arc in Image 5 3 6 1 2 x 3-D Computer Vision CSc 83020 – Ioannis Stamos

28. Slope-Density Function y H(Ψ) HISTOGRAM Lines ψ s Ψ x π 2π Arcs 3-D Computer Vision CSc 83020 – Ioannis Stamos

29. Slope-Density Function y H(Ψ) HISTOGRAM Lines ψ s Ψ x π 2π Arcs H(Ψ) shifts as objects rotates. H(Ψ) wraps around 3-D Computer Vision CSc 83020 – Ioannis Stamos

30. y Fourier Descriptors Find: Ψ(s) Define: Φ(s)= Ψ(s)-(2πs)/P P: Perimeter. 2π: Period of Φ(s) ψ s x Φ(s) is a Continuous, Periodic function. Fourier Series for Periodic Functions: Fourier Coefficients: Φk’s capture shape information Match shapes by matching Φk’s Use finite number of Φk’s

31. Fourier Descriptors Input Shape Reconstruction Power Spectrum # of coefficients Φk Note: Reconstructed shapes are often not closed since only a finite # of Φk’s are used.

32. B-Splines • Piecewise continuous polynomials used to INTERPOLATE • between Data Points. • Smooth, Flexible, Accurate. x2 Spline X(s) x1 s=2 x0 s=1 Data Point xi s s=0 x N BASIS FUNCTIONS We want to find X(s) from points xi Cubic Polynomials are popular: COEFFICIENTS

33. B-Splines Bi(s) has limited support (4 spans) i+2 s: i-2 i-1 i i+1 Each span (i->i+1) has only 4 non-zero Basis Functions: Bi-1(s), Bi(s), Bi+1(s), Bi+2(s) Bi+1(s) Bi(s) Bi+2(s) Bi-1(s) s: i i+1 t=0 t t=1

34. B-Splines 4/6 Bi(s) Bi+1(s) Bi-1(s) Bi+2(s) 1/6 i i+1 t=0 t t=1 Cubic Polynomials 3-D Computer Vision CSc 83020 – Ioannis Stamos

35. B-Splines 4/6 Bi(s) Bi+1(s) Bi-1(s) Bi+2(s) 1/6 i i+1 t=0 t t=1 Cubic Polynomials 3-D Computer Vision CSc 83020 – Ioannis Stamos

36. B-Splines If we compute v0, … vN => continuous representation for the curve. 3-D Computer Vision CSc 83020 – Ioannis Stamos

37. B-Splines We have our N+1 data points: And in matrix form: We can solve for the vi’s from the xi’s Boundary condition for closed curves: v0=vN, v1=vN+1.

38. B-Splines So, for any i we can find: xi(t) t xi xi+1 t=0 t=1 Note: Local support. Spline passes through all data points xi. B-Spline demo: http://www.doc.ic.ac.uk/~dfg/AndysSplineTutorial/BSplines.html

39. Snakes Elastic band of arbitrary shape. Located near the image contour. 3-D Computer Vision CSc 83020 – Ioannis Stamos

40. Snakes – Implementation Discrete version: sum over the points. How can we minimize e? Greedy minimization: For each point compute the best move over a small area. 3-D Computer Vision CSc 83020 – Ioannis Stamos

41. Snakes – Implementation Discrete version: sum over the points. How can we minimize e? Greedy minimization: For each point compute the best move over a small area. Set βi=0 for points of high curvature (corners) 3-D Computer Vision CSc 83020 – Ioannis Stamos

42. Snakes – Implementation Discrete version: sum over the points. How can we minimize e? Greedy minimization: For each point compute the best move over a small area. Set βi=0 for points of high curvature (corners) Stop when a user-specified fraction of points does not move. 3-D Computer Vision CSc 83020 – Ioannis Stamos

43. Synthetic Results Real Experiment Snakes demo: http://www.markschulze.net/snakes/ 3-D Computer Vision CSc 83020 – Ioannis Stamos

44. Region Representation Spatial Occupancy Array 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 • Easy to implement. • Large Storage area. • Can apply set operations (unite/intersect). • Expensive for matching! 3-D Computer Vision CSc 83020 – Ioannis Stamos

45. Quad Trees Efficient encoding of Spatial Occupancy Array using Resolution Pyramids. Black: Fully Occupied. White: Empty. Gray: Partially Occupied. 3-D Computer Vision CSc 83020 – Ioannis Stamos

46. Quad Tree Level 0 Level 1 Level 2 Level 3 Quad Tree Generation: Start with level 0. If Black or White, Terminate. Else declare Gray node & expand with four sons. For each son repeat above step. NW NE SW SE 3-D Computer Vision CSc 83020 – Ioannis Stamos