Neeraj K. Kanhere Dr. Stanley T. Birchfield Dr. Wayne A. Sarasua

1. Vehicle Segmentation and Tracking in the Presence of Occlusions Neeraj K. Kanhere Dr. Stanley T. Birchfield Dr. Wayne A. Sarasua Clemson University

2. Introduction Traffic parameters such as volume, speeds, turning counts and classification are fundamental for… Transportation planning Traffic impact of land use Transportation engineering applications (e.g. signal timing) • Intelligent Transportation Systems (ITS)

3. Why computer vision? Different types of sensors can be used to gather data: Radar or laser based sensors Inductive loop detectors Video Camera (with Computer Vision techniques) • No traffic disruption for installation and maintenance • Covers wide area with a single camera • Provides rich visual information for manual inspection

4. Why tracking? Current systems use localized detection within the detection zones which is prone to errors when camera placement in not ideal. Tracking enables prediction of a vehicle’s location in consecutive frames Can provide more accurate estimates of traffic volumes and speeds Potential to count turn-movements at intersections Detect traffic incidents

5. Related research Region/Contour Based Computationally efficient Good results when vehicles are well separated 3D Model Based Large number of vehicle models needed Limited experimental results Markov Random Field Good results on low angle sequences Accuracy drops by 50% when sequence is processed in true order Feature Tracking Based Handles partial occlusions Good accuracy when sufficient features are tracked from entry region to exit region

6. Factors to be considered High-angle Mid-angle More depth variation Occlusions A difficult problem Planar motion assumption Well-separated vehicles Relatively easy

7. Estimation of 3-D Location Grouping segmented #1 segmented #2 segmented #3 Overview of the approach Background model Offline Calibration Frame-Block #1 Feature Tracking Frame-Block #2 Frame-Block #3 Counts, Speeds and Classification Block Correspondence and Post Processing

8. ` Background model and calibration Adaptive time domain median filtering for background Calibration provides mapping from scene to image Use scene features to estimate correspondences Lane widths Truck heights Approximate calibration is good enough for counts

9. frames Multiple frames are needed for motion information Tradeoff between number of features and amount of motion Typically 5-15 frames yield good results Processing a frame-block Overlap Block # n Block # n+1 #features in block #frames in block

10. Frame differencing Partially occluded vehicles appear as single blob Effectively segments well-separated vehicles Goal is to get filled connected components

11. Box-model for vehicles Road projection using foreground mask Works for orthogonal surfaces Estimation using single frame camera vehicle Road plane

12. Selecting stable features Shadows, partial occlusions will result into wrong estimates Planar motion assumption is violated more for features higher up Select stable features, which are closer to road Use stable features to re-estimate world coordinates of other features

13. Estimation using motion • Estimate coordinates with respect to each stable feature • Choose coordinates which minimized weighted sum of euclidean distance and trajectory error Rigid body under translation Estimate coordinates with respect to each stable feature Select the coordinates minimizing weighted sum of Euclidean distance and trajectory error P : Feature with unknown coordinates Q : Stable feature R : Backprojection on road H : Backprojection at maximum height 0 : First frame of the block t : Last frame of the block Δ : Translation of corresponding point

14. Affinity matrix Each element represents the similarity between corresponding features Three quantities contribute to the affinity matrix Euclidean distance (AD), Trajectory Error (AE) and Background- Content (AB) Normalized Cut is used for segmentation (Shi, Malik) Number of Cuts is not known

15. Incremental normalized cuts We apply normalized cut to initial A with increasing number of cuts For each successive cut, segmented groups are analyzed till valid groups are found Valid group: meets dimensional criteria Elements corresponding to valid groups are removed from A and process repeated starting from single cut Avoids specifying a threshold for the number of cuts

16. Correspondence over blocks Formulated as a problem of finding maximum weight graph Nodes represent segmented groups Edge weights represent number features common over two blocks

17. Results

18. Results

19. Conclusion • A novel approach based on feature point tracking • Key part of the technique is estimation of 3-D coordinates • Results demonstrate the ability to correctly segment vehicles even under severe partial occlusions • Vehicle count, speeds and classification (car or heavy vehicle) data can be easily obtained for tracked vehicles Future Work • Robust block-correspondence • Tracking vehicles at intersections • Automatic calibration by detecting lane markings • Explicit shadow suppression

20. Questions ?

21. Thank you !