IMAGE-DEPENDENT SPATIAL SHAPE-ERROR CONCEALMENT

1. IMAGE-DEPENDENT SPATIAL SHAPE-ERROR CONCEALMENT Professor L.S. Dooley Dr. F.A. Sohel Dr. G.C. Karmakar

2. Presentation outline • What is error concealment (EC) • Traditional vs the image dependent EC • Importance and applications • The proposed EC technique • Experimental results • Conclusion

3. Error concealment • There may be error in the received data • They are exacerbated, due to • Data are compressed • Predictive and variable length coding • Error concealment is to mask the errors • Postprocessing • At the decoder end • Perceptually minimise the effect of the error NOT recovery of the data

4. The big picture

5. Traditional vs image dependent EC Traditional Image dependent

6. Applications • Shape is used as metadata to define an object • Sketch based queries • Hyperlinked video and TV • Content based retrieval using wireless communications • Eg. Mobile

7. Literature Broad classification • Intraframe • Interframe • Intraframe shape error concealment • Maximum a posteriori based estimation on a Random Markov field [1] • Bezier curve based EC [2,3] • Hermite curve based [4]

8. Image dependent shape EC (ISEC) Contour extraction Edge based [has been used in this paper] Vertex based Shape elements

9. Contour extraction Binary shape Object Extracted contour Lost shape blocks

10. Contour coupling Assumptions The contour does not intersect itself and it is closed! Contour coupling is performed In terms of the minimum Euclidean distance [We have now incorporated image information in coupling]

11. Contour recovery • Rubberband function • Detects the edge based on image intensity gradient in a region of interest (RoI) • Takes four parameters • v1 and v2 two endpoints of the RoI, the starting and end points of the recovered contour segment determined from coupling • wd is the width of the RoI • s is a scaling parameter, sets the local or the global view.

12. Contour recovery

13. Contour recovery • Steps • Define the RoI • For all pixels in the RoI calculate the image gradient • Form a directed Graph using the pixels (as vertices) and the gradients (as weight). • Employ the Dijkstra’s algorithm to find the shortest path between the source and destination vertices.

14. Contour recovery • The parameter width (wd) has an important role in computation complexity and the error concealment performance. • Smaller width means a faster but may be less accurate masking • Lagrangian multiplier method for determining the optimal wd • An alternative fast approach has also been proposed

15. Contour recovery Fast width calculations

16. Recovered shape

17. Results ISEC BC HS

18. Results ISEC BC HS

19. Results DLR – Data Loss Rate

20. Results

21. Results with both image and shape loss

22. Conclusion • Proposed a new philosophy: Shape EC using image information • Both optimal and fast width calculation of the rubberband function are proposed • Experimental results warrant its superiority over the existing EC techniques • Produces comparable results when the image information is not available for the concealment process.

23. References [1] S. Shirani, B. Erol, and F. Kossentini, "A concealment method for shape information in MPEG-4 coded video sequences," IEEE Trans. Multimedia, 2(3), pp. 185-190, 2000 [2] M.-J. Chen, C.-C. Cho, and M.-C. Chi, "Spatial and temporal error concealment algorithms of shape information for MPEG-4 video," IEEE Trans. Circuits Syst. Video Tech., 15(6), pp. 778-783, 2005. [3] L. D. Soares and F. Pereira, "Spatial shape error concealment for object-based image and video coding," IEEE Trans. Image Proc., 13(4), pp.586-599, 2004. [4] G. M. Schuster, X. Li, and A. K. Katsaggelos, "Shape error concealment using Hermite splines," IEEE Trans. Image Proc., 13(6), pp. 808-820, 2004.