Hacking the Wiimote and Wii Fit to Help People With Disabilities

1. Hacking the Wiimote and Wii Fit to Help People With Disabilities Rob “roB3AR” Rehrig Josh “Grungy” Marks Larry “Cuddles” Aiello

2. Our Plan • Provide more interface options for people with disabilities • Infrared Tracking • Head to mouse tracking • Gesture recognition • Weight shift tracking • Mouse Clicking • Scrolling • Programmable personal options

3. Why We Chose the Wii Platform • Wiimotes are inexpensive ($40) • Great infrared tracking capabilities • Up to 4 sources • Bluetooth capabilities • Easy to relay information to computer • Existing libraries that access wiimote data • wiiuse.net

4. Introduction to the Wiimote • Nintendo’s controller for the Wii system • Infrared camera in the front • Determines direction Wiimote is moving • Position calculation • Sensor Bar is just 2 clusters of IR LEDs • Wiimote triangulates IR sources to determine position • Track up to 4 IR sources

5. Wiimote Tech: Hardware • Wiimote hardware supports 100 Hz refresh rate • From www.digitalstrategist.com:

6. Hardware: Infra Red • Standard IR camera resolution of 1024 x 768 pixels • Similarly priced webcams are typically 640 x 480 Picture from Wiibrew.org

7. Hardware: Blue-Tooth • Blue-Tooth • Broadcom 2042 chip • Not 100% compatible with HID standard Picture from Wiibrew.org

8. Head Tracking: IR LED Setup • LED Used – Sharp GL4100 • +/- 90 degree viewing angle • Energy efficient – only 1.5V and 50mA needed

9. Head Tracking: IR LED Circuit

10. WiiAssist Tech: Setup

11. WiiAssist Tech: Software • Wiiuse C library – the backbone • Single Threaded Non blocking • Simply coded • One main c++ program that creates 2 threads • 1 thread for wiimote activity (coded in c) • 1 thread for wii fit board activity (coded in c)

12. WiiAssist Tech: Head Tracking • Attach an infrared source to the head • Enables head movement to mouse positioning • Wiimote tracks IR positioning on head • Relays the coordinates to create mouse positioning • 3 Dimensional positioning • Wiimote gives an x and y coordinate to an IR LED • With 2 Wiimotes, one can add a 3rd dimension (z) • x and y are used for mouse movement • z is the distance from the screen

13. Position Wiimotes on the side of the screen • Angle Wiimotes about 22.5° horizontally and vertically • First find the left and right positioning angles • Wiimotes create a 45° angle • Scale the resolution from 0-1024 or 0-768 to 0-45

14. Use angles and • width to calculate • value • With we calculated and then the x coordinate.

15. Use more geometry to calculate z and y coordinates • Since we have adjacent side (x) and angles (Θ1 and Θy) • z and y are opposite sides • z and y are the distance from the 0° area (hypothetically in inches)

16. WiiAssist Tech: Balance Board • 16 bit Pressure sensors in the feet • 1 on each corner (4 total) • Calibrate initial pressure • Returns pressure displacements

17. Balance Board: Error C2065* ? • Wii Fit Board • Thanks to Feesh! On the Wiiuse forums • Helped add wiiboard support • Added Features: • Right / Left Click and Scrolling • SendInput function • Gesture Program Trigger • Windows Hook *bonus points for identifying the Microsoft error code

19. WiiAssist: Teh Future • Smoother mouse control • Moving Average filters to cancel out “noise” • Better integrated Gesture Mapping • Sign language recognition • More user interface capabilities and controls • Auto calibration of wiimotes • Re-design physical Wii Fit Board setup • A simpler design (less wires and power used)

20. Throwin Out the Thanks to these websites and people • http://www.wiili.org • http://www.wiibrew.org • http://www.wiiuse.net • Lazy Foo – Teaching us SDL and multithreading • Johnny Chung Lee - Inspiration • Feesh! – wiiboard help • Nintendo • Independent hackers and reverse engineers

21. Check Us Out! (we’re hot) http://www.cvorg.ece.udel.edu/ http://wiiassist.blogspot.com/

22. Wii love you all! Thank You For Coming!