1 / 16

[ cvPONG ]

[ cvPONG ]. A 3-D Pong Game Controlled Using Computer Vision Techniques. Quan Yu and Chris Wagner. [Project Idea]. <> 3-D adaptation of the classic Pong game <> Score by bouncing the ball past the opponent’s paddles <> Avoid letting the ball go past your paddles.

Download Presentation

[ cvPONG ]

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. [cvPONG] A 3-D Pong Game Controlled Using Computer Vision Techniques Quan Yu and Chris Wagner

  2. [Project Idea] • <> 3-D adaptation of the classic Pong game • <> Score by bouncing the ball past the opponent’s paddles • <> Avoid letting the ball go past your paddles • <> Calculate the user’s paddle positions based on the positions of the user’s hands and head • <> Total of 3 Paddles • <> Implementation: • <> Unity3D + EmguCV (a OpenCV wrapper)

  3. skin detection paddle camera user

  4. [Methods Used] • <> Background Subtraction - Chris • <> Differentiate user from background • <> Skin Detection – Chris, Quan • <> Find skin pixels in image • <> Erosion & Dilation - Chris • <> Clean image • <> Finding Components - Quan • <> Find head and hands

  5. [Background Subtraction] • <> Use background subtraction to obtain mask region for skin detection • 1. Background Subtraction 1 • <> Compare object pixels’ intensities in capture frame with those of previously captured background image • <> How do we choose a good T?

  6. [Background Subtraction] • <> Use background subtraction to obtain mask region for skin detection • 2. Mixture of Gaussians • <> Each pixel modeled by a mixture of K Gaussian distributions • <> Different Gaussians represent different colors • <> Mixture weights determined by time proportions that colors stay in scene • <> Learns probable background colors are the ones which stay longer and are more static • <> Implemented in OpenCV: BackgroundSubtractorMOG()

  7. [Background Subtraction] <> Use background subtraction to obtain mask region for skin detection 2. Mixture of Gaussians An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection P. KaewTraKulPong and R. Bowden

  8. [Skin Detection] • <> Apply skin detection in masked area to locate user’s hands and head • <> Elliptical Boundary Model • <> Convert RGB color space to normalized 2-D chrominance space by eliminating intensity (handles varying illumination conditions) • <> Most non-skin chrominances concentrate on a single point (gray point) • <> Want skin chrominance classification distribution to avoid overlapping this point (false positives) • <> Skin chrominance distribution fits nicely into a skewed normal distribution towards the gray point

  9. [Skin Detection] <> Apply skin detection in masked area to locate user’s hands and head <> Elliptical Boundary Model Chrominance Distribution Single Gaussian Model Elliptical Boundary An Elliptical Boundary Model for Skin Color Detection Jae Y. Lee and Suk I. Yoo

  10. [Finding Components] • <> Skin detection gives binary image in which we can find three largest components (2 hands, head) • 1. Erode then dilate image • <> Removes small components and noise • 2. Calculate contour boundaries of remaining components • <> Use OpenCV method, findContours • 3. Calculate bounding box around three largest contours • 4. Take center of bounding boxes as position of user’s paddles

  11. [Demo]

  12. [Problems] • <> Trade-off between accuracy and performance • <> Shadows/Illumination • <> Glasses! • <> Similarly colored objects in foreground • <> Speed – 8 fps in debug mode • <> Erosion/Dilation removes too many skin pixels • <> Low-fidelity between frames • <> EmguCV is not completely compatible with Unity3D • <> …

  13. [Outcome and Lessons Learned] <> A controlled environment is better than a good algorithm in skin detection <> Vision-based input is a more fun experience than the keyboard/mouse <> Speed/interactivity is important in video games <> Simple background subtraction does not work well <> Simple skin detection works pretty good <> Noise is hard to remove

  14. [Future Work] <> Incorporate MoG background subtraction to prune skin detection areas <> Utilize shape analysis/face detection to separate hands/head <> Smoother paddle movement between frames <> GPU programming to improve performance <> Add more game elements

  15. [Thank you!] Questions, comments?

More Related