Presented By Motion Capture Group: Azadeh Jamalian Ata Naemi Sa'ed Abu-Alhaija Sunghoon Ivan Lee

1. SensIT Technology Presented By Motion Capture Group: Azadeh Jamalian Ata Naemi Sa'ed Abu-Alhaija Sunghoon Ivan Lee

2. Overview • Introduction • System Overview • Hardware unit • Image Processing • Modeling and Animation • Results • Financing • Applications and Marketing • Future Missions • Question Period • Demonstration Welcome to our Presentation!

3. Introduction Study dance movements Robotic Medicine Our Mission: Study dance movements Sports

4. System Overview Image Processing Unit Image Processing Unit Animation Unit • Real time • User friendly • Easy to customize for different applications • Compatible with other systems • Low cost

5. System Overview Image Processing Unit Animation Unit

6. Hardware Block Diagram Microcontroller PIC16LF84A 4 Turn on One LED 4-16 Decoder Sel • Turn on LED’s for 10 frames/sec • Timing issues • Components from local vendors and engineering department

7. Hardware Components • Microcontroller (PIC16LF84A) • - Assembly Language • - Watch Dog Timer (WDT) • - Low power (1mA), Sleep mode (3.5μA) • Regulator (LM317) • - Regulate 3.3V power supply off a 9V battery • - Power off operation as well as staple WDT • Infrared LED’s (KIE-7305) • - 1.6V, draw a current of 2mA • - Each LED is on for 50 msec

8. Test Models • Angle Sensitivity • Problem solving as project progressed • Use high efficient IR LED’s

9. System Overview Image Processing Unit Animation Unit

10. Software Components • Low Level Design • - Capture Control Unit (CCU) • - Image Processing Unit (IPU)

11. Capture Control Unit (CCU) • Three main tasks • 1. Adjusting Phase Difference • 2. Capturing Image • 3. Convert Raw Image into RGB

12. Image Processing Unit (IPU) • Calculate position of IR light source in the frame • Placing film in front of the web camera • - Film absorbs all visible light • - Only IR lights penetrate the film

13. IPU- Challenges • Synchronization with Hardware • Distorted Images and Virtual Images

14. Synchronization Frequency of IR sensors and frame rate can be identical Tb IR Freq. t FrameRate t Tb

15. Synchronization Frequency of IR sensors and frame rate can be identical Tb IR Freq. t Tphase FrameRate t Tb

16. Unexpected Images • Distorted Images & Virtual Images • Solutions: • - Intensity (changing threshold value) • - Width & Height

17. System Overview Image Processing Unit Animation Unit

18. Data Flow Diagram Captured Frames Calculated Coordinates of the LED’s X and Y values Animated Model

19. Animation Unit (AU) • Software: • -C++ (OpenGL) • Main Tasks: • -Reads the coordinates from • an external file • -Assign the coordinates of • each joint and record them • in a file • -Model 2D motion in real time

20. AU- Challenges • Synchronization with IPU • - Timing • - Start Point • - Missing Points • Simulation and Modeling • - Coordinate Systems • - Matrix Transformation and Rotation

21. Simulation Algorithm- Cont. It’s not that easy to think smart!

22. AU- User Interface

23. Results • Frame Rate: 10 frames/sec • Max Motion Frequency: 2Hz • Max Distance from camera: 1m • Precision: 100% (Clear angle of sight) • Accuracy: ± 10%

24. Finance Estimated Cost: Available Funding: $400 from EUSS Actual Cost:

25. Supplementary Application SensIT Mouse: Tracks the motion of finger

26. Marketing • Different applications: • - Art • - Robotic • - Medicine • - Sports • - Game Industry • Competitors: • - Electronic Arts Canada • - C4 Digital Entertainment • - PhoeniX Technologies Incorporated

27. Future Improvements Known Limitations: - Captures the motion only in 2D - Line of sight - Angle of sight - Limited range of monitoring - Interference from other IR sources - Frequency limitation of object motion Improvements: - Capture the motion in 3D - Multiple marking - Enhance the speed

28. What We Learned “If you’re not sure what makes the problem Guess what causes it, Make it worse, Try to solve it, If it didn’t work, try another source!”

29. Conclusion • Our system: • - Research based project • - Proof of concept to design a low cost • motion capture system • - Initial step to designing a 3D motion capture system • In future: • - Unifying music and dance • - Future research opportunities • - Future Coop positions

30. Acknowledgements Thanks a ton! Special Thanks to: Dr. Lucky One Mr. Steve Whitmore Dr. Pavaneh Dr. Patrick Leung Dr. Ivan Bajic Dr. Payandeh Mr. Vinay Kotamraju Mr. Amir Masoud Niroumand Mr. Brad Oldham

31. Questions ?

32. IPU- Cont. Points to make here • Camera Recognizes the IR as a white light • True white color pixel in 24-bit RGB gives • [R,G,B] = [255,255,255] • Only Use one of R,G,B matrices

33. IPU Algorithm

34. IPU Algorithm

35. IPU Algorithm

36. IPU Algorithm

37. Simulation Algorithm - One LED blinks twice indicating the start point - An array to hold the coordinates Useful for applications that track the motion of a specific object  However, not smart enough to detect the position of various joints in real time