Fast Pattern Matching

1. Fast Pattern Matching

2. Fast Pattern Matching:Presentation Plan • Pattern Matching: Definitions • Classic Pattern Matching Algorithms • Fast pattern matching algorithms

3. Pattern Matching: The problem of locating a specific pattern inside raw data. Pattern: A collection of strings described in some formal language. Pattern Matching:Definition

4. Image Pattern Matching:Definition • Pattern Matching: • Finding occurrences of a particular pattern in an image. • Pattern: • Typically a 2D image fragment. • Much smaller than the image.

5. Pattern Matching: Variations • Detecting a set of patterns in a single image: • A set of distinct patterns. • A set of transformations of a single pattern. • Finding a particular pattern in a set of images: • Example: frames in a video sequence.

6. Pattern Matching:Main Tasks • Search in Spatial Domain: • The pattern may appear in any location in the image. • Search in Transformation Domain: • The pattern may be subjected to any transformation (in a transformation group).

7. d( - ) ≥ 0 Image Similarity Measures • Image Similarity Measure: • A function that assigns a nonnegative real value to two given images. • Small measure  high similarity • Can be combined with thresholding.

8. Image Similarity Measure: The Euclidean Distance • Pros: • Computed fast by convolution. • Allows reducing the representations of linear spaces using Principal Component Analysis (PCA) approaches. • PCA preserves Euclidean distance. • Cons: • Not in accord with human perception. • May change drastically when a small transformation is applied.

9. Pattern Matching: Applications • Image Compression • Video Compression • Optical Character Recognition (OCR) • Medical Image Registration • Fingerprint Identification • …

10. Pattern Matching:Real Time Applications • Robot Vision • Vehicle tracking • Surveillance • Industrial Inspection: • Automatic Part Inspection • Automatic Circuit Inspection

11. Fast Pattern Matching • Pattern Matching: Definition • Classical Pattern Matching Algorithms • Fast pattern matching algorithms: • Search in Spatial Domain • Search in Transformation Domain

12. The “Naïve” Approach • Scan the entire image, pixel by pixel. • For each pixel, evaluate the similarity between its local neighborhood and the pattern.

13. Naïve Approach using Euclidean Distance • Given: • k×k pattern P(x,y) • n×n image I(x,y) • For each pixel (x,y), we compute the distance: • Complexity:

14. Fixed Over all pixels: • Convolution can be applied rapidly using FFT. • Complexity: Improvement: FFT

15. Naïve and FFT Approaches: Performance Performance table for a 1024×1024 image, on a 1.8 GHz PC: Far too long for real-time applications!

16. Normalized Grayscale Correlation • NGC: • A similarity measure, based on a normalized cross-correlation function. • Maps two given images to [0,1] (absolute value). • Identical images are mapped to 1. Given two images I and T, NGC is computed as follows:

17. + = Pattern Matching using NGC • Pattern Matching using NGC: • Create a correlation map: • Shift the pattern over the image. • For each position, compute NGC(pattern,window). • Find peaks in the correlation map.

18. Pattern Matching using NGC: Evaluation • Pros: • Very accurate: • By interpolating the correlation map, a pattern can be located in 1/16 pixel accuracy. • Invariant to linear changes in brightness. • Cons: • Computationally expensive: • Matching k×k pattern with n×n image takes O(n2k2). • Limited tolerance to variations in scale, orientation and illumination.

19. Multi Resolution Approach Pattern Matching using a Gaussian Pyramid First suggested by: Burt & Adelson (1984)

20. Multi Resolution Approach: Scale Invariant Search • The same pattern is searched in each pyramid level. • problems: • Computationally expensive. • Limited scale resolution. Adelson, Anderson, Bergen, Burt, Ogden, “Pyramid methods in image processing” (1984)

21. Fast Pattern Matching • Pattern Matching: Definition • Classical Pattern Matching Algorithms • Fast pattern matching algorithms: • Search in Spatial Domain • Search in Transformation Domain

22. Gradient Descent Search in an Image Pyramid A technique for fast search in the spatial domain Suggested by: Maclean & Tsotsos (2000) MacLean, Tsotsos, “Fast pattern recognition using gradient-descent search in an image pyramid” (2000)

23. Find candidates using thresholding. • For each candidate, estimate location in next level. • Refine location by correlation gradient descent, using estimated location as starting point. Iterate search untiltop-level Multi Resolution Schemes:Gradient Descent Search • Create pyramid for pattern and image. • Compute NGC between top level image and pattern. MacLean, Tsotsos, “Fast pattern recognition using gradient-descent search in an image pyramid” (2000)

24. Gradient Descent Search: Evaluation • Pros: • Full NGC is performed only in top level (small). • The pattern pyramid is created off-line. • Cons: • All inherent NGC limitations: • Only 8-10% invariance to scale and rotation. MacLean, Tsotsos, “Fast pattern recognition using gradient-descent search in an image pyramid” (2000)

25. Code only motion vectors for each block Divide a frame into blocks In the next frame, search each block within its surrounding area Heuristics: Focus of Attention • Assumptions on the location of the pattern within the image. • Used for motion estimation in video compression.

26. Pattern Matching using Projection Kernels A new technique for fast search in the spatial domain Suggested by: Y. Hel-Or and H. Hel-Or (2002) Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

27. Motivation • Euclidean distance as a similarity measure. • Fast computation of a lower bound on the distance. • Fast rejection of non-matching windows.

28. || - ||2 Distance Measure in Sub-space • Given: • k×k pattern p. • k×k image window w. • Both w and p can be represented as vectors in Rkxk: Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

29. w p u Distance Measure in Sub-space (cont.) • Assume that p and w are not given. • However, we know their projection values onto a unit vector u. • From the Cauchy-Schwarz Inequality: Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

30. w v p u Distance Measure in Sub-space (cont.) • If we project p and w onto two unit vectors u and v, we can tighten the lower bound: Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

31. v w p u Distance Measure in Sub-space (cont.) • By projecting on a set of mutually orthonormal vectors U: • When U forms a basis, LB = Euclidean distance. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

32. Natural Images w p u u1 Choosing Projection Kernels • A good choice of projection kernels U can expedite the distance calculations. • U should have high probability of being parallel to the vectors p - w. • The first few kernels should capture a large proportion of the distance. • Projection on U should be fast to apply. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

33. The Walsh-Hadamard Kernels • The first 28 kernel vectors. • Displayed in order of increasing spatial frequency. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

34. Projection Kernels: Walsh-Hadamard • Properties: • Fast Transform calculations: • The basis vectors contain only 1. • Transform calculations require only integer additions and subtractions. • First few vectors capture high proportion of image energy. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

35. The Walsh-Hadamard Kernels (cont.) Lower bound on distance between pattern and windowvs.Number of projection kernels used Walsh-Hadamard Standard basis Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

36. Efficient Transformation Computation • The fast transformation is due to: • Computation of one window is exploited in its neighbor. • WH kernels have a recursive structure: • Projection on one kernel is exploited in the next kernel. • Projection onto the first few WH kernels is usually sufficient. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

37. A node:Contains intermediate computations The root:Contains the original signal + - + + - + + - - + + + - + - + - - + + + -- + + + + - - - - + + + + The i’th leaf:Contains a vector of projection values of all signal windows onto the i’th WH kernel + - + - + - + - + - + - - + - + + - - + - + + - + - - + + - - + + + -- + + -- + + -- -- + + + + + + -- - - + + + + + + + + The Walsh-Hadamard Tree (1D) A tree for computing projections onto first 8 WH basis vectors: Tree height = log8 Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

38. + + - + + + - + + + + + + + - + - + + + + + - - + + - + - + + -- The Walsh-Hadamard Tree (2D) • In 2D: • The projections are performed in a similar manner. • Height of the tree: log(k)  2log(k). • Number of leaves: k k2 Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

39. The Pattern Matching Algorithm 1) Projectall image windows on the first WH kernel. • Find a lower bound on the distance between the pattern and each window. 2) Reject all windows whose lower bound exceeds a given threshold. 3) Projectonly the remaining windows on the next kernel. • Tighten the lower bounds of the remaining windows. Iterate 4) Return to step (2), unless either: • All k kernels have been processed. • The number of remaining windows reached a predefined value. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

40. Performance • Performance table for a 1024×1024 image and a 32×32 pattern. • Performed on a 1.8 GHz PC. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

41. Example Initially:65536 candidates Image (256×256) Pattern (16×16) Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

42. Example (cont.) After the 1st projection:563 candidates Image Pattern Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

43. Example (cont.) After the 2nd projection:16 candidates Image Pattern Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

44. Example (cont.) After the 3rd projection:1 candidate Image Pattern Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

45. Tolerance to Noise Results of the algorithm when applied to a noisy image: Original Noise Level = 40 Detected Patterns • Requires a large rejection threshold: More candidates Longer running time Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

46. + + - + + + - + + + + + + + - + - + + + + + - - + + - + - + + -- Illumination Invariance • The DC of all windows is given by projection on the first WH kernel. • We can disregard the DC by skipping the first kernel. • The rest of the process continues as before. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

47. Illumination Invariance (cont.) • Results of the algorithm (without DC): Illumination gradient added Original Image Detected patterns • Only 5 projections were performed. Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

48. P. Matching using Projection Kernels: Summary • Fast projection computations. • Fast rejection of non-pattern windows. • Invariant to: • Noise • Illumination variations Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

49. Fast Pattern Matching • Pattern Matching: Definition • Classical Pattern Matching Algorithms • Fast pattern matching algorithms: • Search in Spatial Domain • Search in Transformation Domain

50.  kxk P=  kxk T(α)P= Search in Transformation Domain • Represent a k×k pattern P as a point in kxk: • Let T(α)P be a transformation T(α) applied to P: • T(α)P for all α forms an orbit in kxk: T(1)P T(0)P T()P T(2)P P