Foundations of Information Visualization: Human Perception and Visual Processing

1. Lecture 2 • Information Visualization Intro – Recap • Foundation in Human Visual Perception • Sensory vs. Cultural • Attention – Searchlight Model • Stages of Visual Processing • Luminance & Color Channels • Pre-Attentive Processing • Mapping Data to Display Variables

2. Goal of Information Visualization • Use human perceptual capabilitiesto gain insightsintolarge data setsthat aredifficult to extractusing standard query languages • Support Exploration • Look for structure, patterns, trends, anomalies, relationships • Provide a qualitative overview of large, complex data sets • Assist in identifying region(s) of interest and appropriate parameters for more focussed quantitative analysis • Abstract and Large Data Sets • Symbolic • Tabular • Networked • Hierarchical • Textual information

3. Information Visualization - Problem Statement • Scientific Visualization • Show abstractions, but based on physical space • Information Visualization • Information does not have any obvious spatial mapping • Fundamental Problem How to map non–spatial abstractions into effective visual form? • Goal Use of computer-supported, interactive, visual representations of abstract data to amplify cognition

4. Student Videos – Essence of Information Visualization • Copy the following URL into Browser window: • http://www.scils.rutgers.edu/~aspoerri/Teaching/InfoVisResources/student_videos/ • and Right click on hyperlink for the name below and use “Save As …” download avi file to computer • Phil Bright • http://www.scils.rutgers.edu/~aspoerri/Teaching/InfoVisResources/student_videos/bright.avi • Carlos Carrero • http://www.scils.rutgers.edu/~aspoerri/Teaching/InfoVisResources/student_videos/carrero.avi • Daveia Thomas • http://www.scils.rutgers.edu/~aspoerri/Teaching/InfoVisResources/student_videos/thomas.avi

5. Approach • 1 Foundation in Human Visual Perception How it relates to creating effective information visualizations • 2 Understand Key Design Principles for Creating Information Visualizations • 3 Study Major Information Visualization Tools • 4 Evaluate Information Visualizations Tools • 5 Design New, Innovative Visualizations

6. Human Visual System – Overview • Sensory vs. Cultural • Attention – Searchlight Model • Stages of Visual Processing • Luminance & Color Channels • Pre-Attentive Processing • Mapping Data to Display Variables

7. Sources • Information Visualization • Perception for Design • Colin Ware • Academic Press, 2000 • As well as: • Marti Hearst (Berkeley) • Christopher Healey(North Carolina)

8. Sensory vs. Cultural

9. Sensory vs. Cultural (cont.) • Visualization = Learned Language ? • Meaning of Symbol = Created by Convention • If true, choice of visual representation arbitrary • Semiotics = Study of Symbols and how they convey Meaning • Choice of Visual Representation Matters • Outlines Object outline and object itself excite similar neural processes Visual cortex designed to detect continuous contours • Similar perceptual illusions / blindness in humans and animals • Not all diagrammatic notations are equal • Most visualizations are Hybrids • Learned conventions and hard-wired processing

10. Physical World Structured • Well-Defined SurfacesObjects have mostly smooth surfaces • Temporal PersistenceObjects don’t randomly appear/vanish • Light travels in Straight Linesreflects off surfaces in certain ways • Law of Gravity

11. Our Premise • Sensory Representations Tap into Perceptual Power of Brain Without Learning • Sensory Representations Effective because well matched to early stages of neural processing • Understanding without training • Perceptual Illusions Persist Mueller-Lyon Illusion (off by 25-30%)

12. Attention – Searchlight Model

13. Attention – Searchlight Properties • Searchlight Size varies with • Data density • Stress level • Attention Operatorswork within searchlight beam • Attention = Tunable Filter • Eye movements 3/sec– series of saccades • Popout Effects(general attention) • Segmentation Effects(dividing up the visual field) •  Guide Attention

14. Stages of Visual Processing • 1 Rapid Parallel Processing • Feature Extraction: orientation, color, texture, motion • Transitory: briefly held in an iconic store • Bottom-up, data-driven processing • 2 Serial Goal-Directed Processing • Object recognition: visual attention & memory important. • Slow and serial processing • Uses both short-term memory and long-term memory • More emphasis on arbitrary aspects of symbols • Different pathways for object recognition & visually guided motion • Top-down processing

15. Parallel Processing • Orientation • Texture • Color • Motion • Detection • Edges • Regions • 2D Patterns • Serial Processing • Object Identification • Short Term Memory 5 ± 2 = 3 to 7 Objects Parallel Processes  Serial Processes a

16. Visual Angle

17. Two Point acuity (0.5 min) Acuities Vernier Super Acuity (10 sec)

18. Contrast Spatial Freq. Spatial Frequency Acuity Need Sufficient Contrast for Fine Details

19. 1 0 0 8 0 6 0 4 0 2 0 5 0 3 0 1 0 1 0 3 0 5 0 D i s t a n c e f r o m F o v e a ( d e g . ) Acuity Distribution

20. Scale Matters

21. Luminance “channel” • Extracts Surface Information • Discounts Illumination Level • Discounts Color of Illumination • Mechanisms 1 Adaptation 2 Simultaneous Contrast

22. Luminance is not Brightness • Luminance = physical measure • Brightness = perceived amount of light • Eye sensitive over 9 orders or magnitude • 5 orders of magnitude (room – sunlight) • Receptors bleach and less sensitive with more light • Takes up to half an hour to recover sensitivity • Eye is NOT a light meter Designed to detect CHANGES Not good for detecting Absolute Values Extremely sensitive to Differences & Changes

23. Simultaneous Contrast

24. Edge Detection

25. Luminance for Shape-from-Shading

26. Color Trichromacy Three cones types in retina

27. 1 0 0 8 0 6 0 4 0 2 0 4 0 0 5 0 0 6 0 0 7 0 0 W a v e l e n g t h ( n m ) Cone Sensitivity Functions – Blue / Green / Red a

28. Color Implications • Color Perception is Relative • Sensitive to Small Differences • hence need sixteen million colors • Not Sensitive to Absolute Values • hence we can only use < 10 colors for coding

29. Rapid Visual Segmentation Only about six categories Color = Classification Color helps us determine type

30. 12 Colors for labeling Color Coding Large areas = low saturation Small areas = high saturation

31. Luminance Channel Detail Form Shading Motion Stereo Chromatic Channels Surfaces of Things Labels Categories (about 6-10) Red, green, yellow and blue are special (unique hues) Channel Properties – Take Home Messages  More Important

32. Color - Take Home Messages • Use Luminance for Detail, Shape and Form • Use Color for Categorization - few colors • Minimize Contrast Effects • Strong colors for small areasContrast in luminance with background • Subtle colors for large areas

33. Pre-Attentive Processing • Some Visual Properties Processed Pre-Attentively • No need to focus attention • Pre-Attentive Properties Important for Design of Visualizations • Can be perceived immediately • Can mislead viewer • < 200 - 250ms • Eye movements = at least 200ms • Some processing can be done very quickly  Implies low-level processing in parallel

34. Segmentation by Primitive Features • How many areas ?

35. 0 8 0 2 8 0 8 5 0 8 0 8 3 0 8 0 2 8 0 9 8 5 0 - 8 0 2 8 0 8 5 6 7 8 4 7 2 9 8 8 7 2 t y 4 5 8 2 0 2 0 9 4 7 5 7 7 2 0 0 2 1 7 8 9 8 4 3 8 9 0 r 4 5 5 7 9 0 4 5 6 0 9 9 2 7 2 1 8 8 8 9 7 5 9 4 7 9 7 9 0 2 8 5 5 8 9 2 5 9 4 5 7 3 9 7 9 2 0 9 Pre-Attentive Processing • How many 3s ?

36. 0 8 0 2 8 0 8 5 0 8 0 8 3 0 8 0 2 8 0 9 8 5 0 - 8 0 2 8 0 8 5 6 7 8 4 7 2 9 8 8 7 2 t y 4 5 8 2 0 2 0 9 4 7 5 7 7 2 0 0 2 1 7 8 9 8 4 3 8 9 0 r 4 5 5 7 9 0 4 5 6 0 9 9 2 7 2 1 8 8 8 9 7 5 9 4 7 9 7 9 0 2 8 5 5 8 9 2 5 9 4 5 7 3 9 7 9 2 0 9 Color  Pre-Attentive (Pops out) • How many 3s ?

37. Orientation and Size - Gabor Primitives

38. 9 0 0 7 0 0 5 0 0 3 6 1 2 N u m b e r o f d i s t r a c t o r s Pre-Attentive Experiment • Number of irrelevant items varies • Pre-attentive 10 msec per item or better. • Decision = Fixed Timeregardless of the number of distractors Preattentive a

39. Pre-Attentive Processing-Color

40. Pre-Attentive Processing-Orientation

41. Pre-Attentive Processing-Motion

42. Pre-Attentive Processing-Size

43. Pre-Attentive Processing-Simple shading

44. Pre-Attentive – Summary

45. Conjunction (does not pop out)

46. Compound features (do not pop out)

47. Example: Conjunction of Features Viewer cannotrapidly and accurately determine if target (red circle) is present or absent when target has two or more features, each of which are present in the distractors. Viewer must search sequentially.

48. Laws of Pre-Attentive Display • Must Stand Outin Simple Dimension • Color • Simple Shape= orientation, size • Motion • Depth

49. Pre-Attentive Channels • Form orientation/size • Color • Simple Motion/Blinking • Spatial, Stereo Depth, Shading, Position

50. Pre-Attentive Demo • Pre-Attentive Demo by Christopher Healey • Target = Red Circle • Distractors • blue circles (colour search) • red squares (shape search) • blue circles and red squares (conjunction search)