Turn angle function and elastic time series matching

1. Turn angle function and elastic time series matching Presented by: Wang , Xinzhen Advisor: Dr. Longin Jan Latecki

2. Agenda • Introduction • Turn angle function • Time series matching • Conclusion • Future work

3. Introduction • Data set: 1400 images (70 classes * 20 objects) Data Preprocessing 1400 land mark sequences

4. Introduction An example: Original image Landmark sequence Time series

5. Introduction • Time sequence matching Shape Matching Sequence Matching

6. Introduction • Problem definition: Given a query q and a distance function d, find m nearest neighbors of q by calculating the distance using d. In our case, m is equal to 40. Distance function d: a = (x1 ,x2 , ...xn) b = (y1 ,y2 , ...yn)

7. Tangent space representation • Shape description in tangent space Problem Step function presentation Simplified contour

8. Turn angle function • Modification to tangent space rep. Rotation (turning angle) Scaling (normalization) Starting point (double length)

9. Landmark sequence • From time series to land mark sequences Step I : compare each point with its left neighbor Step II : compare each point with its left and right neighbors • Disadvantages (ex: loss of information)

10. Landmark sequence matching • Step I : Align the highest peak of the query with every peak of the object, and then align other peaks and valleys of the query accordingly. • Step II: Calculate the Euclidean distance between the peaks/valleys of query and object. As we move query along the object, we have Euclidean distance for each alignment. A smallest Euclidean distance identifies the optimized alignment. • Step III: In the optimized alignment, we introduce a penalty distance if either query or object has extra peaks or valleys.

11. Landmark sequence matching An example: Aligning query peaks and valleys with object (one optimized alignment) Query’s peaks and valleys Object sequence (doubled)

12. Landmark sequence matching Another example : query has extra peaks and valleys Back

13. Landmark sequence matching • Penalty distance if the query has extra peaks/valleys, a penalty distance is added to the Euclidean distance between the query and object. The penalty distance is calculated by the sum of Euclidean distances between the unmatched peaks/valleys to the closest matched peaks/valleys in the query. See example If instead the object has extra peaks/valleys, a penalty distance is calculated by the sum of Euclidean distances between the unmatched peaks/valleys to the closest matched peaks/valleys in the object.

14. Some experimental results • Query: The first object in the 1st class • Search for 40 nearest neighbors in the whole dataset. • The top 40 matches found. • Retrieval Rate : 100%

15. Back

16. Definition Retrieval Rate: Since we have the prior knowledge about those objects within the same class as the query object, we can define the retrieval rate of matching as : RetrievalRate = N / 20 ( N: number of objects in the top 40 matches that belong to the same class as the query object)

17. Part Matching--- In a primitive stage • We manually select a significant part of an object, for example the leaves of an apple, and proceed sub-sequence matching and retrieval • Since our query part has only three peaks and three valleys, we define them as LeftMostPeak/Valley, MiddlePeak/Valley, RightMostPeak/Valley. See here.

18. LeftMostPeak/Valley RightMostPeak/Valley MiddlePeak/Valley Back

19. Part Matching • The Matching Scope in object The closest peak/valley to the LeftMostPeak/Valley The closest peak/valley to the LeftMostPeak/Valley

20. Part Matching • Step I : Calculate the Euclidean distance between the peaks/valleys of the query part and object part. Only peaks/valleys fall between the matching scope in the object are considered for matching. • Step II: As we move query part along the object, we have Euclidean distance for each alignment. A smallest Euclidean distance identifies the optimized alignment. • Step III: In the optimized alignment, we introduce a penalty distance if either query part or object part has extra peaks or valleys. The penalty distance calculation would be the same as previous defined.

21. Some experimental results • Query part: The first object in the 1st class • Search for 40 nearest neighbors in the whole dataset. • Retrieval Rate : 60% • False positives

22. False Positives Come pretty early in the 40 matches! Obj626

23. False Positives Looks like the leaf of an apple?

24. Conclusion • It’s feasible to transform image contour data to time sequence. • Landmark sequence can capture the important features of time series. Matching based on it is applicable and promising. • Part Matching brings good result by submitting very limited query information.

25. Future work • Order of Matching (Eliminate crossover) • Combination of global matching with part matching. • Apply the technique on the whole dataset.

26. Thank you