A Robust Scene-Change Detection Method for Video Segmentation

1. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY A Robust Scene-Change Detection Method for Video Segmentation Chung-Lin Huang and Bing-Yao Liao

2. Outline • Introduction • Abrupt Scene-Change Detection • Gradual Scene-Change Detection • Experimental Results • Conclusion

3. Introduction • The main problem of segmenting a video sequence into shots is the ability to distinguish between scene breaks and normal changes that happen in the scene • This paper combines the intensity and motion information to detect scene changes such as abrupt scene changes and gradual scene changes

4. Previous Problems • The two main problems in most existing algorithms • they are threshold-dependent algorithms • they suffer false detection with scenes involving fast camera or object motion. • This paper proposes a scene-change detection algorithm with three contributions • Relaxing threshold selection problem • higher detection rate (scene change should not be missed) • lower false alarm rate

5. ABRUPT SCENE-CHANGE DETECTION • Method • Measurement of the Changes Between Frames • Pixel-Based Difference • Histogram-Based Difference • Static Scene Test • Scene Transition Classification • Detection Algorithm

6. Detection Algorithm • First phase • locate the highest and the second highest peaks of DCimage difference in the midst of the sliding window, and then calculate the ratio n between the first and second peaks • Second phase • Histogram Measure • Static Scene Test (Most of the false alarms declared by the histogram detector are due to sudden light changes, while the edge information is more or less invariant to these changes) • Scene Transition Nhigh Nlow genuine Ambiguous No Scene Change

8. Measurement of the Changes Between Frames • Pixel-Based Difference: • Histogram-Based Difference: (X2 Test) Where Cx and Cy are the DC image of frames X and Y

9. Color Histogram • Efficient representation • Easy computation • Global color distribution • Insensitive to • Rotation • Zooming • Changing resolution • Partial occlusions • Disadvantage • Ignore spatial relationship • Sensitive in illumination changes • Choose illumination-insensitive color channels

10. Example • Color space selection & quantization • Use RGB channels • Each channel is divided into 2 intervals • Total number of bins = 23 = 8 • H(I): Color histogram for Image I • H1 = (7, 7, 7, 7, 9, 9, 9, 9) • Image 1 has 7 pixels in each C1 to C4, and 9 pixels in each C5 to C8

11. Static Scene Test • Define • all objects present in the scene exhibit rather small motion compared to the frame size, and global movement caused by the camera is slow and smooth. • Method • Edge Detection • Edge Dilation • Result • The transition of two consecutive frames with covering ratio larger than a predefined threshold is considered as a non-static or dynamic scene.

12. Example of edge detection Edge Detection Edge Dilation ( r=3 )

13. Scene Transition Classification • Transition Type • 1) static scene to static scene • 2) dynamic scene to static scene or vice versa • 3) dynamic scene to dynamic scene • Dynamic-to-dynamic transition usually indicates continuous object or camera motion, rather than a real scene change

14. Gradual Scene-Change Detection • Gradual Scene-Change • Dissolve • Fade-In ( X = 0 ) • Fade-Out ( Y = 0 ) • Why not easy to detect • Camera and object motions always introduce a larger variation than a gradual transition. Scene X to Y In Duration T

15. Intensity Statistics Model • Normal Case • For any frames near the reference frame, their dissimilarity measure almost increases exponentially with their distance • Gradual Transition • The dissimilarity increases linearly with their distance during the transition • After the transition is over, the difference measures are randomly distributed

16. Normal Sequence Gradual Transition Seed : the beginning frame of a gradual transition

17. N-distance measure • the difference measure generated by comparing a frame with itself and its successive ( N – 1 ) frames • Ideal model of the N-distance measure

18. Gradual Scene-Change-Detection Algorithm • 1) N-distance measure • 2) Difference operation • 3) Low-pass filtering: Implement a simple low-pass filter to keep the low frequency segments and to remove the high-frequency segments

19. Gradual Scene-Change-Detection Algorithm • If the number of zero crossings between frame k and frame l (k and l is zero crossing frame) • Zero-crossing rate calculation: • larger than a threshold : high frequency fragment • else : low frequency fragment (gradual scene-change segment) • a local “score” record mechanism • Scorei(q) q=0, 1, 2 …, N-1 • High frequency fragment : Scorei(q) = 0 • Low frequency fragment : Scorei(q) = 1

20. N-Distance Measure Difference operation Local Score

21. Gradual Scene-Change-Detection Algorithm • a global “track” record mechanism • Track(p) p = 1, 2, 3, …,L • After every N-distance Measure of framei , we can get the local Scorei(q) • Accumulate the Score record to the Track record • the total number of frames in the video sequence

22. Improve Gradual Scene-Change-Detection Algorithm • To develop a fast seed-searching process • we select one from every S consecutive frames for N-distance measure. • Since gradual scene change does occur in segment 3 only, we need to ignore the scores in segment 1 due to correlation behavior of the reference frame and its neighboring frames. • The correlated distance in segment 1 is C

23. EXPERIMENTAL RESULTS (1)

24. EXPERIMENTAL RESULTS (2) Performance Measure Performance Result

25. EXPERIMENTAL RESULTS (3)

26. Conclusion • This method avoided the false alarms by using the validation mechanism. • It also proves that the statistical model-based approach is reliable for gradual scene-change detection. • Experimental results show that a very high detection rate is achieved while the false alarm rate is comparatively low.