CS6534 Guided Studies “Crowd Flow Analysis”

1. CS6534 Guided Studies“Crowd Flow Analysis” Supervised by Dr. Hau-San WONG Prepared by Kam-fung YU (51150118)

2. Background & challenge • Video Surveillance System are widely used for monitoring • Performance is good as the number of object for detection is small (Spatial variation is small) and ; • The change over time is small (Temporal variation is small) BUT • A challenge for crowd flow • The number of objects is in the order of 102~103 • The change of the scene is very fast

3. Related works • Based on Tracking of Individuals • Shape and Color Model of Individuals • Trajectories of Points • Boundary Contour • xt Slices of Spatio-temporal Video Volume • People Counting in the Crowd

4. Shape & color model of individuals 3D human model • Models human shape by using 3D model • Data-driven Markov chain Monte Carlo (DDMCMC) • Iterate an optimized solution T. Zhao et. al., “Bayesian Human Segmentation in Crowded Situations”, IEEE CVPR03, 2003.

5. Trajectories of points • Similar method with Shape & Color Model • Use some simple feature, such as corner of an object, to extract points probabilistically • Clustering the points into independently moving entities, cluster Shape and Color Model Trajectories of Points G. Brostow et. al., “Unsupervised Bayesian Detetcion of Independent Motion in Crowds”, IEEE CVPR, 2006.

6. Boundary contour • Use of low-interest points to detect the object clustering • Select by the high temporal and spatial discontinuity • Outline the object by joining edges Sample Scene Clustered object P. Tu et. al., “Crowd Segmentation through Emergent Labeling”, In ECCV Workshop SMVP, 2004

7. Xtsclies of spatio-temporal video volume • Scan interesting lines over a certain frames, xt-slice • Use the Hough transform to detect movement in the xt slices 5 corresponding xt slices Sample Scene with 5 lines Hough transform P. Reisman, “Crowd Detection in Video Sequences”, IEEE Intelligent Vehicles Symposium, 2004.

8. People counting in the crowd • Clustering of some feature points by their motion • Estimate the number of people by the number of cluster A result of clustering on two video scene V. Rabaud et. al., “Counting Crowded Moving Objects”, IEEE CVPR, 2006

9. Limitations on tracking of individuals • Involes Iteration • Convergence Decease as Number of Objects Increase • Large Computational Time • High of Computational Power • Difficulty to implement on Real Time Monitoring System

10. Our approach • Proposed by Saad Ali & Mubarak Shah in 2007 • Individual Flow → Global Optical Flow • Tracking Individuals → Measuring Global Quantities • Using Fluid Dynamics to treat the problem • Global Quantities:Finite Time Lyapunov Exponents Field (FTLE), Lagrangian Coherent Structures(LCS) • Expect a Higher & Faster Algorithm in Performance

11. Algorithmic outline

12. Optical flow • 16x16 size block • Displacement vector x • p frames for 1 mean field • q mean field for 1 block mean field S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

13. Algorithmic outline

14. Flow map • Launch a set of particles over the optical flow field • Solve a flow map for a time period T = pq frames • Interpolation a cubic velocity equation by 4th order Runge-Kutta-Fehlberg algorithm (RK4) • x, y are used to record the x and y coordinate at each initial position launched after time Flow map of x-particle Flow map of y-particle S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

15. Algorithmic outline

16. Ftle Field • Compute the four spatial derivates • Plug into the Cauchy-Green deformation tensor • The largest finite time Lyapunov exponent with the maxmum eigenvalue max of the tensor and the period T=pq frames FTLE Field Plot S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

17. Algorithmic outline

18. lcs FTLE Field Plot S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

19. segmentation • This process involves two stages • Cut spatially into different region by the ridges in FTLE • Use the Lyapunov divergence to decided two segment merge or not 1st stage 2nd stage S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

20. Algorithmic outline

21. Flow instability detection • Flow instability is defined as the change in the number of flow segments with respect to time Current segment New segment S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007

22. Capabilities • Capable for monitoring thousands of objects simultaneously • Get rid of number of people constrain • Capable for monitoring flow in any orientation • Obtain same result under any rotation • Capable for new segment detection over time • Locate the increase or the decrease of segments over time

23. Potentials • Potential for flow control or city design • Making immediate decision for crowd flow • Facilitate on the planning of city streets, traffic flow, overhead, bridges and passageways • Potential for flow pattern recognition • Extraction of various flow pattern • Flow pattern solution space construction for a given static scenery • Flow pattern bases finding

24. limitations • Limitation on crowd density • Degraded as crowd density is low • Worse at only a number of objects • Limitation on a large number of many-fold dynamics flow • Too many segments (too noisy) on the scene • Hard to merge segment • Limitation on a rapid unstable flow • Hard to retrieval information from rapid changing flow • Too slow to capture the information

25. Further suggestions • Find out the critical crowd density for an acceptable performance • Finding out a method that can undergo segmentation under a noisy domain • Designing a rapid flow capturing algorithm • Finding out the possible flow patterns on given static scenery • Find out the flow patterns solution space and bases

26. conclusion • In this guided study, we studied about various kinds of methods and the Lagrangian Dynamics in solving the crowd segmentation problem. • We also realized the capabilities, potentials and limitations . • We finally suggested some possible direction for future studies.

27. references • Z.N. Li, M.S. Drew, “Fundamentals of Multimedia”, NJ: Pearson Education Hall, 2004 • P.E. Mattison, “Practical Digital Video with programming examples in C”, NY: John Wiley & Sons Inc, 1994 • L. Perko, “Differential Equations and Dynamical Systems 3rd Ed.”, NY: Springer, 2001 • Intel Corporation, “Open Source Computer Vision Library, Reference Manual”, USA: Intel Corporation, 2001 • S. Ali, M. Shah, “A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis”, CVPR May, 2007 • S. C. Shadden, “Lagrangian Coherent Structures – Analysis of time-dependent dynamical systems using finite-time Lyapunov exponents”,Available Online : http://www.cds.caltech.edu/~shawn/LCS-tutorial/, Last update: 15th April, 2005 • P. Reisman, “Crowd Detection in Video Sequences”, IEEE Intelligent Vehicles Symposium, 2004. • T. Zhao et. al., “Bayesian Human Segmentation in Crowded Situations”, IEEE CVPR03, 2003. • P. Tu et. al., “Crowd Segmentation through Emergent Labeling”, In ECCV Workshop SMVP, 2004. • G. Brostow et. al., “Unsupervised Bayesian Detetcion of Independent Motion in Crowds”, IEEE CVPR, 2006. • D. Yang et. al., “Counting People in Crowds with a Real-Time Network of Simple Image Sensors”, ICCV, 2003. • V. Rabaud et. al., “Counting Crowded Moving Objects”, IEEE CVPR, 2006. • E. Rosten and T. Drummond, “Machine learning for high-speed corner detection”, Europe Conference on Computer Vision, May 2006. • C. Tomasi and T. Kanade, “Detection and tracking of point features”, Technical Report CMU-CS-91-132, Carnegie Mellon University, April 1991. • S. Ali, “Crowd Flow Segmentation & Stability Analysis”, Available Online : http://www.cs.ucf.edu/~sali/Projects/CrowdSegmentation/index.html , Last visited: 30th Nov, 2008

28. Department of Computer Science City University of Hong Kong Thank you