Eye Tracking and Gaze Detection For Computer Mouse Conrol

1. Eye Tracking and Gaze DetectionFor Computer Mouse Conrol By: Ryan Sellers Justin Williams

2. Original Design Criteria • Webcam used to detect eye motion • IR sensor that determines user distance from monitor • Less intrusive than electrooculograms • Real-time analysis of eye movement • Easy setup with no calibration or headgear • Inexpensive

3. Outline Final Block Diagram Original Block Diagram

4. Fabrication • Webcam Modifications • PSoC to PC Communication • IR LED Circuit

5. Modifying the Logitech CS500 Webcam Removal of IR filter and addition of IR LEDs

6. PSoC Evaluation Board • Programmed with PSoC Designer • Powered by USB from Computer • Uses RS232 Protocol • Handles Communication Between Computer and Hardware RS232 PSoC To IR LEDs and Sensor Power from USB LED Circuit

7. IR Sensor RS 232 IR Sensor

8. Testing Curve From Data Sheet Piece Wise Curve Based on Data

9. Testing

10. IR LED Circuit • Illumination controlled by PSoC • 4 Bit Digital Signal • Ultra Bright IR LEDs

11. Testing

12. Testing • Varying light in room, observed change in ambient light calculations • Used LEDs in dark room with dim screen • At reasonable distances, very little facial illumination • Solutions: • Add parabolic lens to focus light. • Add filter to remove visible light.

13. Software Design • Psoc Interface • Adaptive Lighting • Pupil Tracking • Mouse Control

14. PSoC Interface • Found C++ wrapper class to interface with Serial Com Ports • UART-Universal Asynchronous Receiver/Transmitter • Use driver to emulate RS232 with USB cord with a virtual Com Port • Set up basic ascii protocol

15. Adaptive Lighting • Average of 100 random samples within face region to determine ambient lighting • Average over 5 previous frames • Four intensity levels

16. Testing Intensity = 42 Out = 1000 Intensity = 96 Out = 0100 Intensity = 125 Out = 0010 Intensity = 140 Out = 0001 Above: Intensity vs. Digital Output Right: Verification of 100 samples vs. True Average.

17. Pupil Tracking • Image Processing • Gaussian Blur • Histogram Equalization • Conversion to Grayscale • Methods • Haar Features Eye Detection • Canny Edge Detection and Hough Transform • Convolution with design Kernel • Size: Speed vs. Accuracy • Weights

18. Testing Canny Bottom, Dark, Justin Bottom, Dark, Justin Front, Dark, Ryan Left, Dark, Justin

19. Testing 7x7 Convolution Kernel 5x5 Convolution Kernel Bottom, Dark, Ryan Right, Dark, Justin

20. Final Implementation Bottom Dark Justin Front Light Ryan

21. Mouse Control • Using windows library • Using pupil location and center of mass of the eye, finds the angle at which the eye is looking • Algorithm to find center of mass of the eye • Requires stationary point on face • Using face box • Using eye detection with Haar features • Using green marker on face

22. Testing • Wrote an OpenGL program • Puts a point on the screen • Measures accuracy every half second

23. Algorithm For Determining Center of Eye Use pupil to estimate center of the eye relative to a stationary point on the face Use each new measurement of the pupil to get a better guess for the center of mass of the eye Use the current estimate for the center of mass of the eye to determine gaze direction

24. Recommendations • Product Feasibility • Faster Implementation: Plausible • Robust to Lighting: Plausible • Robust to all Head Motion: Unlikely • Eye to Screen Accuracy: Unlikely • User with Glasses: Unlikely

25. Questions?