Improved Moves for Truncated Convex Models

1. Improved Moves for Truncated Convex Models M. Pawan Kumar Philip Torr

2. Aim Efficient, accurate MAP for truncated convex models V1 V2 … … … … … … … … … … … … … … … … … Vn Random Variables V = { V1, V2, …, Vn} Edges E define neighbourhood

3. Aim Accurate, efficient MAP for truncated convex models lk ab;ik ab;ik = wab min{ d(i-k), M } li wab is non-negative d(.) is convex Vb b;k a;i Va Truncated Linear Truncated Quadratic ab;ik ab;ik i-k i-k

4. Motivation Low-level Vision min{ |i-k|, M} • Smoothly varying regions Boykov, Veksler & Zabih 1998 • Sharp edges between regions Well-researched !!

5. Things We Know • NP-hard problem - Can only get approximation • Best possible integrality gap - LP relaxation Manokaran et al., 2008 • Solve using TRW-S, DD, PP Slower than graph-cuts • Use Range Move - Veksler, 2007 None of the guarantees of LP

6. Real Motivation Gaps in Move-Making Literature Chekuri et al., 2001 2 2 + √2 O(√M) Multiplicative Bounds

7. Real Motivation Gaps in Move-Making Literature Boykov, Veksler and Zabih, 1999 2 2 2 + √2 2M O(√M) - Multiplicative Bounds

8. Real Motivation Gaps in Move-Making Literature Gupta and Tardos, 2000 2 2 2 + √2 4 O(√M) - Multiplicative Bounds

9. Real Motivation Gaps in Move-Making Literature Komodakis and Tziritas, 2005 2 2 2 + √2 4 O(√M) 2M Multiplicative Bounds

10. Real Motivation Gaps in Move-Making Literature 2 2 2 + √2 2 + √2 O(√M) O(√M) Multiplicative Bounds

11. Outline • Move Space • Graph Construction • Sketch of the Analysis • Results

12. Move Space • Initialize the labelling • Choose interval I of L’ labels • Each variable can • Retain old label • Choose a label from I • Choose best labelling Va Vb Iterate over intervals

13. Outline • Move Space • Graph Construction • Sketch of the Analysis • Results

14. Two Problems • Choose interval I of L’ labels • Each variable can • Retain old label • Choose a label from I • Choose best labelling Large L’ => Non-submodular Non-submodular Va Vb

15. First Problem Submodular problem Va Vb Ishikawa, 2003; Veksler, 2007

16. First Problem Non-submodular Problem Va Vb

17. First Problem Submodular problem Va Vb Veksler, 2007

18. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb

19. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb

20. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb

21. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb

22. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb Model unary potentials exactly

23. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb Similarly for Vb

24. First Problem am+1 bm+1 am+2 bm+2 am+2 bm+2 an bn t Va Vb Model convex pairwise costs

25. First Problem Wanted to model ab;ik = wab min{ d(i-k), M } For all li, lk  I Have modelled ab;ik = wab d(i-k) For all li, lk  I Va Vb Overestimated pairwise potentials

26. Second Problem • Choose interval I of L’ labels • Each variable can • Retain old label • Choose a label from I • Choose best labelling Non-submodular problem !! Va Vb

27. Second Problem - Case 1 s ∞ ∞ am+1 bm+1 am+2 bm+2 an bn t Va Vb Both previous labels lie in interval

28. Second Problem - Case 1 s ∞ ∞ am+1 bm+1 am+2 bm+2 an bn t Va Vb wab d(i-k)

29. Second Problem - Case 2 s ∞ ub am+1 bm+1 am+2 bm+2 an bn t Va Vb Only previous label of Va lies in interval

30. Second Problem - Case 2 s ∞ ub M am+1 bm+1 am+2 bm+2 an bn t Va Vb ub : unary potential of previous label of Vb

31. Second Problem - Case 2 s ∞ ub M am+1 bm+1 am+2 bm+2 an bn t Va Vb wab d(i-k)

32. Second Problem - Case 2 s ∞ ub M am+1 bm+1 am+2 bm+2 an bn t Va Vb wab ( d(i-m-1) + M )

33. Second Problem - Case 3 am+1 bm+1 am+2 bm+2 an bn t Va Vb Only previous label of Vb lies in interval

34. Second Problem - Case 3 s ∞ ua M am+1 bm+1 am+2 bm+2 an bn t Va Vb ua : unary potential of previous label of Va

35. Second Problem - Case 4 am+1 bm+1 am+2 bm+2 an bn t Va Vb Both previous labels do not lie in interval

36. Second Problem - Case 4 s ua ub Pab M M am+1 bm+1 ab am+2 bm+2 an bn t Va Vb Pab : pairwise potential for previous labels

37. Second Problem - Case 4 s ua ub Pab M M am+1 bm+1 ab am+2 bm+2 an bn t Va Vb wab d(i-k)

38. Second Problem - Case 4 s ua ub Pab M M am+1 bm+1 ab am+2 bm+2 an bn t Va Vb wab ( d(i-m-1) + M )

39. Second Problem - Case 4 s ua ub Pab M M am+1 bm+1 ab am+2 bm+2 an bn t Va Vb Pab

40. Graph Construction Find st-MINCUT. Retain old labelling if energy increases. am+1 bm+1 am+2 bm+2 an bn t Va Vb ITERATE

41. Outline • Move Space • Graph Construction • Sketch of the Analysis • Results

42. Va Vb Va Vb Previous labelling f’(.) Global Optimum f*(.) Analysis Va Vb Current labelling f(.) QC ≤ Q’C QP

43. Va Vb Previous labelling f’(.) Analysis Va Vb Va Vb Current labelling f(.) Partially Optimal f’’(.) Q’0 ≤ QC ≤ Q’C

44. Va Vb Previous labelling f’(.) Analysis Va Vb Va Vb Current labelling f(.) Partially Optimal f’’(.) QP- Q’0 ≥ QP - Q’C

45. Analysis Va Vb Va Vb Va Vb Current labelling f(.) Partially Optimal f’’(.) Local Optimal f’(.) QP- Q’0 ≤ 0 ≤ 0 QP - Q’C

46. Analysis Va Vb Va Vb Va Vb Current labelling f(.) Partially Optimal f’’(.) Local Optimal f’(.) QP- Q’0 ≤ 0 Take expectation over all intervals

47. QP ≤ 2 + max 2M , L’ L’ M Q* QP ≤ O(√M) Q* Analysis Truncated Linear Gupta and Tardos, 2000 L’ = M 4 L’ = √2M 2 + √2 Truncated Quadratic L’ = √M

48. Outline • Move Space • Graph Construction • Sketch of the Analysis • Results

49. Synthetic Data - Truncated Linear Energy Time (sec) Faster than TRW-S Comparable to Range Moves With LP Relaxation guarantees

50. Synthetic Data - Truncated Quadratic Energy Time (sec) Faster than TRW-S Comparable to Range Moves With LP Relaxation guarantees