Image Sequence Coding by Split and Merge

1. Image Sequence Codingby Split and Merge Patrice Willemin, Todd R. Reed and Murat Kunt Presented by: Idan Shatz

2. Outline • Split and Merge Coding • For Still Images • Split and Merge Coding • For Image Sequence • Conclusions

3. Image Coding • First Generation • Pels coding (prediction or transform) • Second Generation • Object Based coding • Region based coding • Split and Merge Method

4. Split & Merge • Split… • Recursive algorithm • Approximate a region by a 2D polynomial • If (approximation error > threshold) • Split the region into 4 sub-regions • Approximate each sub-region

5. Split Example Original Image Split, depth=1

6. Split Example Original Image Split, depth=2

7. Split Example Original Image Split, depth=3

8. Split Example Original Image Split, depth=4

9. Split Example Original Image Split, depth=5

10. Split Example Original Image Split, depth=6

11. Split Example Original Image Split, depth=7

12. Split Example Original Image Split, Final results Threshold=100,000

13. The threshold 100,000 50,000 20,000 10,000 5,000 2,000 1,000 500 200 100

14. 5,000 100,000 The threshold

15. Split & Merge • An improvement of the Split algorithm • At each step decide between x-split and y-split • produces fewer regions

16. Split Example (2) Original Image Split, depth=1

17. Split Example (2) Original Image Split, depth=2

18. Split Example (2) Original Image Split, depth=3

19. Split Example (2) Original Image Split, depth=4

20. Split Example (2) Original Image Split, depth=5

21. Split Example (2) Original Image Split, depth=6

22. Split Example (2) Original Image Split, depth=7

23. Split Example (2) Original Image Split, depth=8

24. Split Example (2) Original Image Split, depth=9

25. Split Example (2) Original Image Split, depth=10

26. Split Example (2) Original Image Split, depth=11

27. Split Example (2) Original Image Split, depth=12

28. Split Example (2) Original Image Split, depth=13

29. Split Example (2) Original Image Split, Final results Threshold = 100,000

30. Split Example (Comparison) Split – 4 sub regions Threshold = 100,000 # regions = 613 Split – 2 sub regions Threshold = 100,000 # regions = 387

31. Comparison

32. Image Coding • Each region is codes by: • The Border • 2D polynomial coefficients

33. The borders of the regions • BSP-Tree • Binary space partition tree • Tree Nodes: • “X” – mark an X-split • “Y” – mark an Y-split • “0” – mark an Region

34. Regions Borders • Example of BSP-Tree Y X 0 0 Y X 0 0 0

35. Storing a BSP Tree • N regions  2N-1 BSP-Nodes • Storing the tree in preorder Y X 0 Y X 0 Y X 0 0 0 0 0 Y X 0 0 0

36. Image Coding • For N regions • Border (BSP) – N*3[bit] • 2D polynomial coefficients – N*24[bit] • 8[bit] x 3 – gray level. • 8[bit] x 3 x 3 - for RGB.

37. Image Coding • Without Compression • 196,608[Byte] • With Compression • # regions = 387 • 3628[Byte] • Compression Raito: • 1 to 54 Split – 2 sub regions

38. Split & Merge • Merge… • For each region: • 3[bit] border << 54[bit] Coefficients • Reduce the number of regions • Merging neighbor regions with similar coefficients.

39. Merge Example (Comparison) Split – 4 sub regions Threshold = 100,000 # regions = 613 After Merge # region = 411 Split – 2 sub regions Threshold = 100,000 # regions = 387 After Merge # regions = 348

40. Comparison

41. The borders of the regions • The borders are more complex • Simple BSP Tree is not enough • Therefore • BSP Tree • Regions labels

42. Regions Labeling • How many bits are needed for labeling? • ~1000 regions  ~10[bit] • Non-neighbor regions can have the same labeling • ~3[bit] for labeling • Does not depend on the number of regions

43. Image Coding • For N regions made from M BSP-regions • Border (BSP) – M*3[bit] • Labeling – M*3[bit] • 2D polynomial coefficients – N*24[bit] • 8[bit] x 3 – gray level. • 8[bit] x 3 x 3 - for RGB.

44. Image Coding • Without Compression • 196,608[Byte] • With Compression • # BSP regions = 387 • # regions = 348 • 3446[Byte] • Compression Raito: • 1 to 57 (>54) Split – 2 sub regions And merging

45. y t x Image Sequence Coding • Same Idea. • Still Image  (x,y) • Video  (x,y,t) • Split Stage • Decide between x-split, y-split and t-split. • Merge Stage • Same

46. Real Time Transmission • Compress all movie at once • Achieving best compression. • However • can’t be transmitted on-line • Very high complexity • Deal with chunks of 5~25 frames • Compression ratio decreases • Can be transmitted on-line

47. Conclusions • Still Images compression • Achieves good results with good compression ratio. • 2 major defects due to region boundaries: • Disappearance of important boundaries • Border artifacts between regions. • Merge Regions • using the same threshold has little effect on performance • adaptive threshold for achieving a giving compression ratio

48. Conclusions • Image Sequence compression • Achieves good results • At the article: 1 to 200 comp. ratio. • But, compression ratio decreases on high movement scenes. • compression ratio  still images. • Merging regions can be used to balance the buffer size. • Hard to decode at real-time. • Must be coded at frame chunks. • Decoding process is hard to implement in real time environment.

49. Go To… • Still Images • Split • to 4 sub-regions • to 2 sub-regions • Threshold(2) • Compression • Merge • Threshold • Compression • Image Sequence • Conclusions