S C A L E D PATTERN MATCHING

1. SCALED PATTERN MATCHING A.Amir Bar-Ilan Univ. & Georgia Tech A.Butman Holon College M.Lewenstein Bar-Ilan Univ. E.Porat Bar-Ilan Univ.

2. TEMPLATE AERIAL PHOTO SEARCHING FOR TEMPLATES IN AERIAL PHOTOGRAPHS INPUT TASK: Search for all locations where the template appears in the image.

3. local errors Theoretically, need to consider: • Noise • Occlusion • Scaling (size) • Rotation (orientation) We are interested in asymptotically efficient algorithms in pixel space.

4. MODEL • Low Level (pixel level) avoid costly preprocessing • Asymptotically efficient solutions. • Serial, exact algorithms.

5. Results: mismatches AL-88 edit distance, k errors, rectangular patterns edit distance, k errors, half-rectangular patterns AF-95 TYPES OF APPROXIMATIONS Local Errors: Level of detail Occlusion Noise

6. Results: FU-98 ACL-98 TYPES OF APPROXIMATIONS Orientation

7. A B C EVEN WITHOUT ERRORS AND ROTATIONS … DIGITIZING NEWSPAPER STORIES IDEA: Keep dictionary of fonts Search for appearances in all size.

8. 1 2 3 4 5 PROBLEM INHERENTLY INEXACT What if appearance is 1½ times bigger ? What is ½ a pixel ? SOLUTIONS UNTIL NOW: NATURAL SCALES Consider only discrete scales

9. How does one model for real scales?

10. 1 2 3 4 5 6 7 T[1,1] T[1,2] T[1,3] 1 T[2,1] T[2,2] T[2,3] 2 T[3,1] T[3,2] T[3,3] 3 4 T[5,4] 5 6 T[7,7] 7 Step 1:Define grid & pixel centers. Example:Unit pixel array for a 77 array.

11. Step 2:Define scaling. Example:33 array. Scaled To 1⅓

16. Scaled To 1⅓

17. Remark: • We only scale “up” Reasons: • Avoid conceptual problems of loss of resolution. • From “far enough” away everything looks the same.

18. Let P be a mm pattern and T an nn text. How many different scaled patterns of P are there?

19. In fact, can there be two different scaled patterns of P of size kk?

20. Example: 5×5

21. 4 x 1.1 = 4.4 Scaled by 1.1 to 6x6

22. 4 x 1.125 = 4.5 Scaled by 1.125 to 6x6

23. 3 x 1.17 = 3.51 Scaled by 1.17 to 6x6

24. 2 x 1.25 = 2.5 Scaled by 1.25 to 6x6

25. Let P be a mm pattern and T an nn text. How many different scaled patterns of P are there?

26. Claim There are ≤ nm different scaled patterns representing all the occurrences of P.

27. … Proof: Each one has at most m possible matrices representing it mm, (m+1)(m+1), … , nn n-m different possible sizes

28. Proof: Each one has at most m possible matrices representing it Why?

29. Distance = 1

30. … … … … … … … … … (k-½)×(k -½) (k+½)×(k +½) … … … Pattern P scaled to size k×k,

31. 1

32. Therefore There are ≤ nm different scaled patterns representing all the occurrences of P.

33. Algorithm outline for 2-D scaled matching

34. Straightforward Idea • Construct dictionary of O(nm) possible scaled occurrences of P. • Use 2-dimensional dictionary matching algorithm to scan the text in linear time and find all occurrences.

35. … Space and Time Analysis Each one has at most m possible matrices representing it mm, (m+1)(m+1), … , nn Dictionary size O(n3m)

36. Solution • Our idea is to keep the dictionary in compressed form. • The compression we use is run-length of the rows.

37. Run-length aabcccbb a2b1c3b2

38. The compressed dictionary Scaled To 2⅓

39. # of repetitions of row Compressed form

40. # of repetitions of row Size of Array: mxm # of diff. scaled patterns: (n-m) x m Dictionary size: O(nm3)

41. The Idea behind the text searching • For every text location [i,j], we assume that there is a pattern scaled occurrence beginning at that location. • Subsequently, we establish the number of times this row repeats in the text. • This allows us to an appropriately scaled pattern row from the dictionary.

42. T: 1 a10 b c150 … a10 b c150 … 2 . . . . . . . . . . . . . . . P: 1 a b c100 a b c100 2 a10 b c150 … 132 . . . . . . . . . . . . b10 c d150 … 133 . . . . . . . . . . . . . . . 102 a b c100 Example for text searching

43. 1 a10 b c150 … aaaaaaaaaa

44. T: 1 a10 b c150 … a10 b c150 … 2 . . . . . . . . . . . . . . . P: 1 a b c100 a b c100 2 a10 b c150 … 132 . . . . . . . . . . . . b10 c d150 … 133 . . . . . . . . . . . . . . . 102 a b c100 Look in the text location [1,10]

45. T: 1 a10 b c150 … a10 b c150 … 2 . . . . . . . . . . . . . . . P: 1 a b c100 a b c100 2 a10 b c150 … 132 . . . . . . . . . . . . b10 c d150 … 133 . . . . . . . . . . . . . . . 102 a b c100

46. 1 a10 b c150 …

47. 1 a10 b c150 … aaaaaaaaaab

48. 1 a10 b c150 … Look in the text location [1,10] aaaaaaaaaabccccccccccccccc…

49. 22 2½2½ T: 1 a10 b c150 … a10 b c150 … 2 . . . . . . . . . . . . . . . P: 1 a b c100 a b c100 2 a10 b c150 … 132 . . . . . . . . . . . . b10 c d150 … 133 . . . . . . . . . . . . . . . 102 a b c100

50. T: 1 a10 b c150 … a10 b c150 … 2 . . . . . . . . . . . . . . . P: 1 a b c100 a b c100 2 a10 b c150 … 132 . . . . . . . . . . . . b10 c d150 … 133 . . . . . . . . . . . . . . . 102 a b c100 A scale range of [1,1¼)