Trajectory-Based Ball Detection and Tracking with Aid of Homography in Broadcast Tennis Video

1. Trajectory-Based Ball Detection and Tracking with Aid of Homography in Broadcast Tennis Video Xinguo Yu, Nianjuan Jiang, Ee Luang Ang Visual Communications and Image Processing 2007 Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6508 Present by komod

2. Introduction • The ball is the most important object in tennis (and in many kind of sports) • Very challenging problem • Camera motion • presence of many ball-like objects • small size and the high speed of the ball • Object-indistinguishable

3. Introduction • Method • Trajectory-based • the ball is the “most active” object in tennis video • previous work： A Trajectory-based ball detection and tracking algorithm in broadcast tennis video, Proc. of ICIP • Homography • Goal • find projection locations of the ball on the ground • find landing positions

4. Introduction

5. Introduction

7. Feature Point Extraction • Court Segmentation • Find the court color range and paint all the pixels in this range with a single color • find the lines separating the audience from the playing field • detecting the change pattern of color for each row and column of the image • paint the audience area in the court color.

8. Feature Point Extraction • Straight Line Detection • gridding Hough transform • Court Fitting • Detect the net and use it as reference • find the intersection of lines

9. Homography Acquisition • Standard Frame • whose lookat is the cluster center of all lookats of all the frames in the considered clip • The lookat of frame is a point in the real world that corresponds to the center of the frame

10. Homography Acquisition • Disparity Measure of Two Court Images • For i = 1 to 9 • Measure Function • Let Cstd be the court in the standard frame and Ctrn denote the transformed court from the segmented court in frame F • For given H and F

11. Homography Acquisition • Initial Matrix • transforms an image point X' (x1', y2', 1) to a point X (x1, y2, 1) in another image • X = HX‘ • Tuning of Homography • The homograph matrix computed based on feature points • A small hough space enclosing it

12. Homography Acquisition • Tuning procedure • Frame transform

14. Ball Location In Hitting Frame • Hitting frame detection • Find the sound emitted by the racket hitting • M. Xu et al, Creating audio keywords for event detection in soccer video, In Proc. of ICME • Hitting racket detection • Maybe player tracking

16. Ball Candidate Detection • Object segmentation from standard frame • Four sieve are used for non-ball object removal • Court Sieve Θ1 • filter out audience area • filter out court lines • Ball Size Sieve Θ2 • filter out the objects out of the ball-size range • homography from ground model to standard frame • use a range of allowable ball sizes (estimate error) • Ball Color Sieve Θ3 • filter out the objects with too few ball color pixels • Shape Sieve Θ4 • filter out objects out of the range of width-to-height ratio • 2.5 is suggested in previous paper

17. Ball Candidate Detection • Each sieve is a Boolean function on domain Ο(F) • The set of remaining objects is C(F) • C(F) = {o : o ∈O(F), Θi(o)=1 for i = 1 to 4} • Candidate Classification • Three features are use • Size, color, and distance from other objects • The ball-candidates are classified into 3 Categories

19. Candidate Trajectory Generation • No detail explanation in this paper • X. Yu et al, Trajectory-based ball detection and tracking of broadcast soccer video, IEEE Transactions on Multimedia, issue 6, 2006. • Candidate Feature Plots (CFPs) • CFP-y • CFP-l

20. The algorithm is actually works on the CFP-l which are 3-D plots

21. Candidate Trajectory Generation

23. Trajectory Processing • Trajectory Confidence Index • Let T be a candidate trajectory • and λ1,λ2,…,λm, be all properties of trajectory T • confidence index Ω(T)

24. Trajectory Processing • Trajectory Discrimination

25. Trajectory Processing • Ball Projection Location • y = an3 + bn2 + cn + d. • Ball Land Detection • form a ball position function against frame number i, y = f(i) • find the maximum of f '(i) between each pair of hittings

26. Experimental Results • 5 clips • extracted from mpeg2 704x576 • average time for acquiring ball candidates • ALGnew for a frame is 86.15s on a P4/1.7Ghz PC with 512MB RAM • ALGold is 19.21s

27. Experimental Results BPL

28. Experimental Results

29. Experimental Results • average discrepancy of all detected balls from the groundtruth previous result

30. Experimental Results • frames with inserted 3D projected virtual content • Homography in home surveillance video

31. Conclusion and Future Works • The previous algorithm mainly alleviated the challenges raised by causes besides camera motion • The algorithm presented in this paper additionally counteracts the challenges brought to us by the camera motion • The contributions of this paper are two-fold • it develops a procedure to robustly acquire an accurate homograph matrix of each frame • it forms an improved version of ball detection and tracking algorithm • Two future works • evolve the algorithm into an end-to-end system for ball detection and tracking of broadcast tennis video • analyze the tactics of players and winning-patterns, and hence produce rich indexing of broadcast tennis video by making use of the ball position

32. Any Question?

33. Thank You

34. Experimental Results • 7 segments, total 120 s, mpeg1 video, Men’s Final of FRENCH OPEN 2003 back