Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail 12/10/2009

1. Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009

2. Outline • Eye Tracking Device • Application areas • State-of-the-art technology • Eye tracker @NCCUCS 1.0 (wearable) • Eye tracker @NCCUCS 2.0 (remote) • Gaze-based HCI Demo • Conclusions

3. Eye Tracking Device • An eye tracker is a device for measuring eye positions and eye movements. • The most popular variant uses video images from which the eye position is extracted. • Input source: visible spectrum vs. infrared

4. Eye Movements • Eye movements are typically divided into fixations(when the eye gaze pauses in a certain position) and saccades (when it moves to another position). • The resulting series of fixations and saccades is called a scanpath.

6. Web Design (U. of Manchester) Heat map Gaze plot

7. Usability Study:Google Search Evaluation

8. Gaming

9. Driving Behavior [Andrew T. Duchowski]

10. Character Input EyeWrite [Andrew T. Duchowski]

11. Media Research • The Poynter Institute : http://www.poynter.org • Published first eye track study in 1991. • More results published in 2000, 2004 and 2007. • Eyetrack ’07:http://www.poynter.org/content/content_view.asp?id=105035

12. Research Communities • ACM SIG/CHI • Eye Tracking Research and Applications (ETRA) Symposium: http://www.e-t-r-a.org/ • COGAIN: Communication by Gaze Interaction http://www.cogain.org • References: http://www.cogain.org/downloads/

13. State-of-the-art Eye tracking Technology • Hi-Speed (SMI iView X™ Hi-Speed) • Head-Mounted System • Remote tracking • Integrated with LCD monitor

14. Mobile Eye Tracker 1.0 @ NCCUCS scene camera eye camera

15. Key Objectives • Low-cost (NTD 10,000 vs. 1,000,000) • Mobility • Easily customized for specific applications • On-line processing • Sampling rate? • Accuracy?

16. System Architecture Eye image Calibration process Preprocessing 9 pairs of points Pupil detection Calibration Gaze point projection Scene image

17. Pupil Detection (I):extracting feature points

18. Pupil Detection (II): Ellipse fitting using RANSAC

19. Pupil Detection (III):checking fitness measure • Ellipse model: Number of bright vs. dark points inside and out the perimeter

20. Results

21. HCI Application: Eye Scrolling

22. HCI Application: Eye Gaming

23. HCI Application: Eye Typing

24. Mobile Eye Tracker 2.0 @ NCCUCS • Improve the pupil detection algorithm to alleviate corneal reflection problem. • Enhance the accuracy by compensating for head movement. • Construct and test a remote eye tracker. • More HCI applications using the remote eye tracker. • Use the eye tracking device to assist mobile user interface design.

25. Improved Pupil Detection

26. Allowing Head Movement

27. Remote Eye Tracker

28. Experimental Results (Wearable) 1 cm = 38 Pixels

29. Accuracy (Wearable)

30. Experimental Results (Remote)

31. Accuracy (Remote Eye Tracker)

32. Demo: Web Browsing

33. Demo: Photo Viewing

34. Demo: Interactive Story Telling

35. Demo: Tic-Tac-Toe

36. Summary • Eye tracking device • Low cost (<$10,000 NTD) • Head-mounted with mobile functionality • Remote eye tracking • Allow slight head movements • Accuracy: suitable for some HCI applications, image viewing tasks, not high enough for reading • HCI applications

37. http://www.cs.nccu.edu.tw/~whliao/dct/

38. More materials

39. Gaze Path Analysis • Recursive intersection • Find the similarity between two scan paths • Order of scanning is irrelevant • Suitable for processing fixation data • Modified dynamic time warping • Order of scanning is taken into account • Can handle both fixation and saccade data

40. Path 1 Path 2 Recursive Intersection

41. Recursive Intersection: Example

42. Dynamic Time Warping (DTW)

43. Modified DTW (MDTW) time X-axis Y-axis

44. MDTW Result