Robust Segmentation of Freight Containers in Train Monitoring Videos

1. Robust Segmentation of Freight Containers in Train Monitoring Videos Qing-Jie Kong*, Avinash Kumar**, Narendra Ahuja**,Yuncai Liu* **Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign *Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University, Shanghai 200240, China WACV 2009

2. Input • Input : Video of an Intermodal Freight Train captured from a fixed camera with background visible before the train arrives.

3. Video Capture of an Intermodal Freight Train Viewing Volume Inter-modal Train Camera

4. Output • Output : Video with background removed and thus foreground consisting of only the intermodal train. • Main Application : Fast and Automatic calculation of gap lengths between consecutive containers

5. Example: Input and Output Video

6. Major Difficulties Varied Outdoor Imaging Conditions

7. Major Difficulties Different Types of Containers

8. Four Stage Coarse-to-Fine Framework • Stage 1: Detecting Train Region • Stage 2: Removing Background Gap • Stage 3: Detecting Single Stack • Stage 4: Refining Segmentation Result

9. Stage 1: Detecting Train Region Partition of the region Pixel signal in temporal domain Power spectrum of the signal

10. Stage 1: Detecting Train Region A frame in a video Frequency image of the video Histogram of the frequency image

11. Stage 1: Train Region Detection Thresholded result By the morphological operations Final result of the first stage

12. Stage 2: Removing Background Gap Background Model Background Removal Background Update

13. Background Model A background image A sub-region of the background image Histogram of the sub-region

14. Background Removal A frame in a video Result of the recognition Segmentation result

15. Background Update The background between two containers spans the complete background every some frames Splice the detected middle backgrounds to rebuild a new background image The updating calculation happens as soon as the middle background region completes the scan

16. Stage 3: Detecting Single Stack A frame in a video Result after the first two stages blob Segmentation result

17. Stage 4: Refinement of Segmentation Results Background image A window of background Result before refinement Result after refinement

18. Combination of Color Information Do all the processing in Stage 2 to the RGB channels respectively Combine the results of the three channels by the AND operation

19. Experiments Video data: 150 videos Include 1222 containers and a wide range of background conditions: – clear blue sky – bright sunlight – static heavy clouds in the day and evening – moving heavy clouds in the day and evening – rainy day (water on lens)

20. Experiments Success ratio of Stage 1: 96% Success ratio of the last three stages:

21. Experiments

22. Experiments Operation speed – computer : Intel(R) Core(TM)2 Due CPU 2.53-GHz processor and 3.2-GB MHz RAM. – average processing speed: 4 frames per second (fps)

23. Conclusion The proposed method – combines the information in frequency and spatial domain – is robust to varieties of background conditions – can employ videos from un-calibrated cameras – is being integrated into a real time vision system for intelligent train monitoring

24. Thanks to BNSF