1 / 42

JugEm : Software to help the world juggle

JugEm : Software to help the world juggle. Level 4 Project 2013 Presentation David McBrierty. Juggling. The act of continuously tossing into the air and catching (a number of objects) so as to keep at least one in the air while handling the others. The idea….

juro
Download Presentation

JugEm : Software to help the world juggle

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. JugEm: Software to help the world juggle Level 4 Project 2013 Presentation David McBrierty

  2. Juggling The act of continuously tossing into the air and catching (a number of objects) so as to keep at least one in the air while handling the others.

  3. The idea… • Build a piece of software that can be used to teach people how to juggle.

  4. Existing Software • Lots of existing tutorial help out there already….

  5. Existing Software

  6. More Existing Software

  7. Theressomething missing…. • Existing tutorials are not interactive! • Learn by doing!

  8. What is JugEm? • JugEm is an interactive application to help teach people how to juggle.

  9. Initial ideas…. • Use mobile devices to get sensor data • Make users juggle actual devices • Strap devices to users hands • Wii Controllers • Existing project has done this!

  10. Kinect • Kinect Sensor provides an interesting interface

  11. How good is a good juggle? • Lots of lovely physics behind what happens • Claude Shannon’s Uniform Juggle Theorem

  12. Third Party Libraries • Kinect for Windows SDK • Gives access to raw Kinect Data • EmguCVImage Processing • .NET Wrapper around OpenCV library • Microsoft Speech Platform 11 • Allows Kinect Sensor to use voice commands • Built around Microsoft XNA Framework • Framework for making Xbox and Windows software

  13. System Requirements MUST HAVE SHOULD HAVE Detect user throws Calculate Shannon Score Detect user catches Suggestions on better technique Detect peaks Detect drops Track users hand positions Detect 4 Ball patterns Provide report on technique Detect 1,2 and 3 Ball patterns COULD HAVE WOULD LIKE TO HAVE Define patterns using Siteswap Tasks to improve timing Tasks to improve throwing

  14. The good stuff…. • The system goes through 5 stages:

  15. Kinect Data Input • Color Image represented as byte[] • byte[] can easily be converted to an image.

  16. Kinect Data Input • Depth Data provided as DepthImagePixel[] • Each pixel from Color Image has DepthImagePixel for its distance from sensor • Depth data has to be processed before it is useful!

  17. The good stuff…. • The system goes through 5 stages:

  18. Depth Image Filtering • Raw depth data is very noisy, has to be filtered • 76800 pixels filtered (30 times a second) • Image is converted to gray scale (0-255) • All non player pixels are removed • EmguCV Binary threshold applied to results • Anything behind players wrists is removed! • Kinect thinks the balls are part of the player

  19. The good stuff…. • The system goes through 5 stages:

  20. Ball Detection • EmguCV runs Canny Edge Detection on image • Finds edges with specific radius • Need to detect circles and capped ellipses

  21. Screenshot

  22. Ball Color Detection • Uses Kinect Color Image • RGB Color Detection proved to be inaccurate • Hue, Saturation, Value Color Detection instead! • Extra work, but necessary

  23. Frames • A Frame is used to store all the data collected from processing • All the Juggling Balls seen with color, direction and position • Positions of Users hands • Distance from each ball to each hand • Time Frame was seen • Each new Frame is compared to the previous one to find out directions and detect Events.

  24. The good stuff…. • The system goes through 5 stages:

  25. Detecting Events • Two areas around users hands • Hand • Throw

  26. Detecting Throws and Catches • THROW: Ball seen inside Throw Zone and then outside • CATCH: Ball seen outside Throw Zone and then inside

  27. The good stuff…. • The system goes through 5 stages:

  28. Processing Events • System collects the Events generated and processes them once the user is finished • Each type of Juggle has a pattern • System must be able to cope with missed events

  29. Missed Events • Inferring missed Events can be tricky… • Pattern = P, RT, RC, P, LT, LC.

  30. Missed Events • Inferring missed Events can be tricky… • Pattern = P, RT, RC, P, LT, LC.

  31. Missed Events • Inferring missed Events can be tricky… • Pattern = P, RT, RC, P, LT, LC.

  32. Missed Events • Inferring missed Events can be tricky… • Pattern = P, RT, RC, P, LT, LC.

  33. Missed Events • Inferring missed Events can be tricky… • The times of some inferred Events are slightly out • Just sometimes a problem… • Overall effect on time is small.

  34. Juggle Report • After the Events have been analysed and processed • System has all the information it needs for Shannon Score • Display report to the user

  35. Personal Reflection • Learned how to juggle! • Got to work in a new language • Working with Kinect • Working with all the frameworks • Image processing • Easily the biggest bit of work completed to date….

  36. Personal Reflection

  37. fin

More Related