Dynamics of human eye-movements & visual search

1. Dynamics of human eye-movements & visual search 6/12/08--Presentation for RU CBIM center Deborah J. Aks RU-Center for Cognitive Sciences (RuCCs) daks@rci.rutgers.edu Background research:Perceptual dynamics http://aks.rutgers.edu/..

2. Overview • Visual search studies---------------------------- • Tools to study dynamics • Eye-tracking • Stats, time-series analysis • Power laws (PDFs & power spectra) • Sources of (1/f) power laws:SOC, feedback & recurrent models

4. Visual search & tumor detection • Mammograms • x-rays, CT-scans • Ultrasound • MRI…

5. Challenging search task:Is there a tumor in this mammogram? (x, y) = (0,0) (1024, 0) mov (1024, 768) (x, y) = (0,768) Image source: Society for Breast Imaging (SBI)

6. Abnormal Features Image source:Edward J. Delp; Purdue University School of Electrical and Computer Engineering; Video and Image Processing Laboratory (VIPER) West Lafayette, Indiana, ace@ecn.purdue.edu http://bmrc.berkeley.edu/courseware/cs298/fall99/delp/berkeley99.htm http://www.ece.purdue.edu/~ace

7. Human -vs- Computer-aided detection Both use search, feature detection & classification • Human advantage: • Pattern recognition & implicit learning • (Unsystematic) search patterns can be effective • Computer advantage: • Explicit memory • Only biases are those built into algorithm • Thorough & systematic search • No fatigue

8. (0,0) (1024, 0) X,Y - Eye Samples (1 per ms) (x, y) = (0,768) (1024, 768)

9. (0,0) (1024, 0) Saccades (0,768) (1024, 768)

10. (0,0) (1024, 0) Fixations (0,768) (1024, 768)

11. (0,0) (1024, 0) (0,768) (1024, 768)

12. Dynamical toolsTime series (x,y eye-samples) x y

13. Time series (x,y eye-samples) x rt lft up y down

14. Time series Saccades Fixations y x

15. Time series Saccades Fixations

16. Time series Pupil Velocity Eye-samples: Saccade x y Fixation y Acceleration down x rt

17. Velocity y Acceleration Saccades Fixations x

18. Map trajectory of eye scan-paths: • x,y coordinates(position over time) • ------------------------------------------------------- • Saccades & Fixations: • Sequences (xi, xi+1.. xn) & (yi, yi+1.. yn) • Differences (xn – xn+1 ) & (yn – yn+1) • Distance =(x2 + y2)1/2 • Duration(msec, sec…) • Direction = Arctan (y/x)

19. Dynamical tools • Descriptive & Correlational Statistics • Scatter & Delay plots • Probability Distributions (PDFs) • Power spectra (FFT)… • Autocorrelation • Visual Recurrent Analysis (VRA) • Relative Dispersion (SD/M) • Rescaled range (R/S) • Iterated Functions Systems (IFS)

20. Study 1 • What guides complicated eye movements? • Random or non-random process? • Is there (long-range) memory across fixations? • Might neural interactions drive search? • METHOD OF TESTING. • Challenging visual search task

21. Find the upright “T” T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T

22. Eye Fixations Scatter plot of 10,215 eye fixations for the entire visual search experiment.

23. Conventional GLM analyses… What’s the central tendency (i.e., typical scale)? • 24 fixations per trial (on average) • 5 seconds (SD =7 sec) per trial • Mean fixation duration = 212 ms (SD = 89 ms)

24. Scaling across 8 sessions: Changes in fixations over time: • Frequency decreased from 1888 to 657 • Duration increased from 206 to 217 ms. • Position … • xn – xn+1 decreased • yn – yn+1 increased No typical scale!

25. Heavy-tail distributions • Small eye-mvmts are (very) common; large ones are rare! • Power-laws (?) xn - x n+1

26. Scale-free --> Rethinking what we study & measure Power laws! Many # Few Small Large Size (of behavior)

27. PDF’s & Networks A. Kurakin

28. Brain network Edelman, G. & Tononi, G. (2000).A Universe of Consciousness: How Matter Becomes Imagination..

29. Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128

30. Focusing on the dynamic… How does info decay over time?

31. Delay Plot of Fixations yn-vs- y n+1

32. Spectral analysis (e.g., FFT) Characterize frequencies making up time series f a f -2 = 1/ f 2

33. Noisy time series White Pink Brown

34. “Color’ or pattern White 1/f 0 1/f Pink 1/f 2 Brown

35. PowerSpectra of eye samples 

36. Fixations & their 1st differences  = -.6

37. Distance across eye fixations (x2 + y2) 1/2  = -.47

38. Distance across eye fixations (x2 + y2) 1/2  = -.47  = -0.3  = -1.8

39. Preliminary (FFT) results: • Sequence of… • sampled eye positions --> 1/f brown noise • local random walk • Fixation sequence & their differences--> ~1/f pink noise • Subtle long-term memory.

40. Power law (PDFs) & power spectra (FFT) indicates… • Memory • Steepness of the slope (on a log-log scale) reflects.. • Correlation across data points = ‘Colored’ noise • Pink (1/f) • Brown (1/f^2) • Fractal properties: • Scale-free (means  w/ measuring resolution) • Self-similar (statistically) • Critical + flexible + self-organizing (1/f)

41. Ongoing experiments: • What conditions produce 1/f pink noise? • Do power laws change under different conditions? • Structured vs. unstructured contexts? • Simple vs. complex backgrounds? • Do 1/f patterns produce more effective search? • Do power laws change as we learn?

42. Study 2: Search Displays Complexity --->

43. Search Displays w/ scan paths

46. Baseline: Scanning edge

47. Extrinsic vs. Intrinsic influence on search

50. Source of 1/f dynamic? (Major controversy!)