Comparison of complex background subtraction algorithms using a fixed camera

1. Geoffrey Samuel PhD Researcher Intelligent Systems and Robotics Research Group (ISR) Creative Technologies University of Portsmouth Comparison of complex background subtraction algorithms using a fixed camera

2. Intro Background subtraction is a important and vital step for computers to understand and interpreter a real-world scene It allows a computer to ignore a background so to concentrate on a foreground object

3. Hypothesis Each background subtraction algorithm will have its advantages and disadvantages, and that looking and comparing these with a real-world situation, it would be possible to pick one algorithm or a method of combining algorithms to produce a algorithm capable of balancing speed with quality.

4. The Goal Test and evaluate the quality and speed of existing background subtraction algorithms on a complex background with different everyday motions, and to compare the results with those of the extracted “Ground Truth”

5. Complex Background Static Background:- Background does not contain any secondary “unwanted” motion. Controlled environment. Complex Background:- Background contains secondary “unwanted” motion such as the winds effect on trees or blinds. Real-world data.

6. Synthetic Test Data Advantages: • Automatically got the “Ground Truth”. • More control over each test clip. Disadvantages: • Manual frame by frame “Ground Truth” extraction. • Added artefacts from the Chroma keying and compositing.

7. The Experiment To Create a set of synthetic data with the “Ground Truth” To test different motions with each background subtraction algorithm To Compare the results of each algorithm with that of the “Ground Truth”

8. The Motions • 7 everyday motions were chosen: • Drinking • Jogging • Picking up wallet • Scratching head • Sitting down • Standing up • Walking • Each motion started on the left of the screen and concluded on the right.

9. Creating the test scenarios Green Screen Green Screen with actor Back Ground Final Composite “Ground Truth”

10. The Algorithms 50 Back Plate Difference │framei – backplate│>Ts

11. The Algorithms 50 Frame Difference │framei – framei-1│>Ts

12. The Algorithms Approximate median (x = ( framei- framei-1 – framei-2 . . .framei-n ) > Ts ) → {background += 1} → {background -= 1}

13. The Algorithms k Mixture of Gaussians frame(it = μ) = Σi=1ωi,t .ț(μ,o)

14. Measuring the Quality (0,576) (768,576) (0,576) (768,576) (0,0) (768,0) (0,0) (768,0) Compare the Per-Pixel value of each frame with the “Ground Truth”

15. Results - Quality Most correct pixels Most incorrect pixels

16. Results - Quality

17. Results - Speed “Fastest” Algorithm “Slowest “Algorithm

18. Results - Speed

19. Results - Speed ...now ignoring the Mixture of Gaussian speed results

20. Conclusion Backplate difference was the fastest and produce the highest results in 4 out of 7 tests. Frame difference was the ONLY algorithm to correctly remove the complex background, but couldn't correctly identify the foreground element.

21. Conclusion Frame Difference :- Correctly Removed Complex Background Incorrectly Removed inside of Subject Backplate Difference :- Correctly Identified Subject Incorrectly kept Complex Background

22. Taking it further Theory Framework idea: ƒ Frame Difference Backplate Difference Complex background removed A new method that incorporated both the speed of updating to remove the background and yet the knowledge of the background to properly extract the wanted foreground element.

23. Where can this lead? • Application of this technology could be used in: • Games • Surveillance • Mesh reconstruction and silhouette extraction • Various computer vision tasks

24. Any Questions?

25. Acknowledgments UK Engineering and Physical Science Research Council Seth Benton for his Matlab code

26. Thank you for your time Geoffrey.Samuel@Port.ac.uk www.GeoffSamuel.com