Real Time Pattern Detection

1. Real Time Pattern Detection Yacov Hel-Or The Interdisciplinary Center Hagit Hel-Or Haifa University

2. Pattern Detection A given pattern is sought in an image. • The pattern may appear at any location in the image. • The pattern may be subject to any transformation (within a given transformation group). •••

3. Example Face detection in images

4. Why is it Expensive? The search in Spatial Domain Searching for faces in a 1000x1000 image, is applied 1e6 times, for each pixel location. A very expensive search problem

5. P kxk Why is it difficult? The Search in Transformation Domain • A pattern under transformations draws a very complex manifold in “pattern space”: • In a very high dimensional space. • Non convex. • Non regular (two similarly perceived patterns may be distant in pattern space). P Q (Q,P) T()P

6. A rotation manifold of a pattern drawn in “pattern-space” The manifold was projected into its three most significant components.

7. Suggested Approach • Reduce complexity of search using 2 complementary • processes: • Reduce search in Spatial Domain. • Reduce search in Transformation Domain. Both processes are based on a Rejection Scheme.

8. Efficient Search in the Spatial Domain •••

9. ••• The Euclidean Distance

10. Complexity (2D case)

11. Norm Distance in Sub-space • Representing an image window and the pattern as vectors in Rkxk: dE(p,q)= ||p-q||2=|| - ||2 • If p and q were projected onto a vector u, it follows • from the Cauchy-Schwarz Inequality: dE(p,q)  |u|-2 dE(pTu, qTu) q p u

12. q u2 p u1 Distance Measure in Sub-space (Cont.) • If q and p were projected onto a set of vectors [U]: It can be shown that:

13. Natural Images u1 How can we Expedite the Distance Calculations? Two necessary requirements: • Choose projecting kernels [U] having high probability to be parallel to the vector p-q. • Chooseprojecting kernels that are fast to apply.

14. Projecting Kernels: Walsh-Hadamard Following the above requirement we use the kxk Walsh-Hadamard basis vectors • Each window in a natural image is closely spanned by the first few basis vectors. • Can be applied very fast in a recursive manner.

15. The Walsh-Hadamard Kernels:

16. Walsh-Hadamard v.s. Standard Basis: The lower bound for distance value in % v.s. number of standard basis projections, Averaged over 100 pattern-image pairs of size 256x256 . The lower bound for distance value in % v.s. number of Walsh-Hadamard projections, Averaged over 100 pattern-image pairs of size 256x256 .

17. The Walsh-Hadamard Tree (1D case) + - + + - + + - - + + + - + - + - - + + + -- + + + + - - - - + + + + + - + - + - + - + - + - - + - + + - - + - + + - + - - + + - - + + + -- + + -- + + -- -- + + + + + + -- - - + + + + + + + + WH-tree for 1D pattern of size 8

18. The Walsh-Hadamard Tree - Example 15 6 10 8 8 5 10 1 + - + 21 16 18 16 13 15 11 9 -4 2 0 3 -5 9 + - + + - - + + + - + - + - - + + + -- 39 32 31 31 24 11 -4 5 -5 12 + + + + 3 0 5 1 2 7 -4 -1 5 6

19. Properties: • Descending from a node to its child requires one addition operation per pixel. • A projection of the entire image onto one basis vector is performed in a top-down traversal. • A projection of a particular window in the image onto one basis vector is performed in a bottom-up traversal. + • All operations are performed in integers. - + + - + + - - + + + - + - + - - + + + -- + + + +

20. Complexity (1D): • Projecting all windows in the image onto a basis vector requires log k additions per pixel. • Projecting all windows in the image onto l<k basis vectors requires m additions per pixel, where m is the number of nodes preceding the l leaf. • Projecting all windows in the image onto k basis vectors requires 2k additions per pixel. • Projecting a single window onto a single basis vector requires k-1 additions. + - + + - + + - - + + + - + - + - - + + + -- + + + +

21. Walsh-Hadamard Tree (2D): • For the 2D case, the projection is performed in a similar manner where the tree depth is 2log k • The complexity is calculated accordingly. + + - + + + - + + + + + + + - + - + + + + + - - + + - + - + + -- Construction tree for 2x2 basis

22. Pattern Matching algorithm • Iteratively apply Walsh-Hadamard vectors to each window wi in the image. • At each iteration and for each wi calculate a lower-bound Lbi for |p-wi|2. • If the lower-bound Lbi is greater than a pre-defined threshold, reject the window wi and ignore itin further projections.

23. Pattern Matching algorithm - Complexity All windows are projected onto the first kernel : 2logk ops/pixel Only a few windows are further projected using ~2k operations per active window :  ops/pixel Total : 2logk +  ops/pixel

24. Example: Sought Pattern Initial Image: 65536 candidates

25. After the 1st projection: 563 candidates

26. After the 2nd projection: 16 candidates

27. After the 3rd projection: 1 candidate

28. Percentage of windows remaining following each projection, averaged over 100 pattern-image pairs. Image size = 256x256, pattern size = 16x16.

29. Accumulated number of additions after each projection averaged over 100 pattern-image pairs. Image size = 256x256, pattern size = 16x16. Average Number of operations per pixel: 8.0154

30. Example with Noise Original Noise Level = 40 Detected patterns. Number of projections required to find all patterns, as a function of noise level. (Threshold is set to minimum).

31. 35 30 4 25 3 20 2 15 1 % Windows Remaining 0 10 0 5 10 15 5 0 -5 0 50 100 150 200 250 Projection # Percentage of windows remaining following each projection, at various noise levels. Image size = 256x256, pattern size = 16x16.

32. DC-invariant Pattern Matching Illumination gradient added Original Detected patterns. Five projections are required to find all 10 patterns (Threshold is set to minimum).

33. Complexity (2D case)

34. Advantages: • Walsh-Hadamard per window can be applied very fast. • Projections are performed with additions/subtractions only (no multiplications). • Integer operations. • Fast rejection of windows. • Possible to perform pattern matching at video rate. • DC invariant pattern matching. • Extensions: • Other norms. • Multi size pattern matching.

35. Limitations: • 2n2 log k memory size. • Pattern size must be 2m . • Limited to normed distance metrics.

36. Efficient Search in the Transformation Domain

37. Transformation Manifold A pattern Pcan be represented as a point in kxk T()P is a transformation T() applied to pattern P. T()P for all  forms an orbit in kxk T(1)P T(0)P T()P T(2)P P

38. Fast Search in Group Orbit • Assume d(Q,P) is a distance metric. • We would like to find (Q,P)=min d(Q, T()P) P Q T()P (Q,P)

39. Fast Search in Group Orbit (Cont.) P • In the general case (Q,P) is not a metric. Q R • Observation: if d(Q,P)= d(T()Q, T()P) (Q,P) is a metric

40. Fast Search in Group Orbit (Cont.) The metric property of (Q,P) implies triangular inequality on the distances. Q P S

41. 2 T2(i)P  T(i)P Orbit Decomposition • We can divide T(i)P into two sub-orbits: • T2(i)P and T2(i)P’ where P’= T(1) P • In practice T()issampled into T (i)=T(i) , i=1,2,… P’ P (Q,P) Q

42. 2(Q,P) 2(Q,P’) (Q,P) P P P’ P’ T(i)P T2(i)P T2(i)P’ Orbit Decomposition (Cont.) Q

43. 2(Q,P) 2(Q,P’) P 2(P,P’) P’ T2(i)P T2(i)P’ Orbit Decomposition (Cont.) Q Since 2is a metric and 2(P,P’) can be calculated in advance we may save calculations using the triangle inequality constraint.

44. Orbit Decomposition (Cont.) • The sub-group subdivision can be applied recursively.

45. Fast Search - Example