1 / 44

Techniques of Perception with X3D

Techniques of Perception with X3D. Nicholas F. Polys Luciano Pereira Soares. Session Outline. Polys Perception for Design Visualization Soares Immersion and Perception Polys Information-Rich Environments. Perception for Design.

tory
Download Presentation

Techniques of Perception with X3D

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Techniques of Perception with X3D Nicholas F. Polys Luciano Pereira Soares Web3D Symposium 2005

  2. Session Outline • Polys • Perception for Design • Visualization • Soares • Immersion and Perception • Polys • Information-Rich Environments Web3D Symposium 2005

  3. Perception for Design • Using our understanding of the human perceptual systems to guide design • Visual system • Auditory system • Vestibular system • Leverage pre-attentive facilities • Reduce cognitive overhead Web3D Symposium 2005

  4. Background:Information Psychophysics • Edward Tufte, Envisioning Information (1983, 1990) • Jaques Bertin, Semiology of Graphics (1983) • Donald Norman, Cognitive Engineering (1986) • Joseph Goguen, Semiotic Morphisms (2000) • Colin Ware, Perception for Design (2003) Web3D Symposium 2005

  5. Pre-attentive Processing • Involuntary, do not require conscious attention • Parallel • Efficient • Resistant to instruction Web3D Symposium 2005

  6. Visual Fields Fovea Useful Field of View Field of View Field of Regard Web3D Symposium 2005

  7. Frame Rate • Threshold for perceiving continuity: • flicker < 50 Hz • > 24 fps looks smooth & plenty interactive • Flicker & Attention can lead to change blindness (Simmons, 2000) • Browser.getCurrentFrameRate() • Implementing X3DPerFrameObserverScript - public void prepareEvents (){} Web3D Symposium 2005

  8. Graphics and Vision Constructivism: • Bottom-up : internal representations are built out of simple external properties • Top-down : hypotheses, expectations Perceptual processes are influenced by both; top-down processes take over for ambiguous or degraded stimuli Web3D Symposium 2005

  9. Graphics and Vision: bottom-up • Objects – integration, recognition, working memory and reasoning • Patterns – contours, regions, groups; Gestalt • Features – color, texture, stereoscopic depth, shape & form, some aspects of motion Web3D Symposium 2005

  10. Features: Color • Luminance channel (3x spatial accuity) • Red / Green channel • Yellow / Blue channel The spectrum is not a perceptually linear sequence (not pre-attentive)! (Keller 1993; Ware, 2000) Web3D Symposium 2005

  11. Shapes & Appearances • Appearance {} and Materials {} : specular, emissive, and diffuse Colors in RGB, shininess, transparency, ambientIntensity • creaseAngle: shading across polygons edges of the mesh • normals (for shape-dependent lighting control) • colorPerVertex Web3D Symposium 2005

  12. RGB Material {} diffuseColor 0.678, 0.169, 0.07 specularColor shininess Web3D Symposium 2005

  13. Textures • ImageTexture {} with (or without) alpha channels can be applied and mapped to geometry as fixed or animated maps.  • Standard formats: .png, .jpg, • MovieTexture {} • TextureTransform {} … • PixelTexture {} Web3D Symposium 2005

  14. MultiTexture {} Blending operations specified via mode field Base Texture + Lightmap                   = Result Web3D Symposium 2005

  15. Lighting Lighting Nodes: on, intensity, ambientIntensity, color • Pointlight {attenuation} • DirectionalLight {} • Spotlight {direction, beamWidth, cutOffAngle} • AMD 1: SFBool global Web3D Symposium 2005

  16. Features: Depth • Occlusion • Motion Parallax • Linear Perspective • Relative size • Texture & shade gradients • Stereoscopy • Oculormotor cues • Transform {translation rotation} • Head-Up-Display / Imageplane Web3D Symposium 2005

  17. Patterns & Grouping • Gestalt principles • Also: continuation, closure, common fate • Guiding Law of Pragnanz (simplest, most stable configuration) Web3D Symposium 2005

  18. Gestalt principles • Palmer & Rock, 1990– review & update principles; grouping based on perceived proximity in 3D space (not 2D proximity on retina) • Quinlan & Wilton, 1998 – study involving Gestalt conflict; proposed resolution mechanisms Web3D Symposium 2005

  19. Objects • Feature Binding – putting the streams together for internal represenation • color, form, motion • Just in time? • 2.5 D sketch (Marr, 1982) • Geons (Biederman, 1993) Web3D Symposium 2005

  20. Attention • Pop out effects ‘stand out’ in some simple dimension (conjunctions don’t): • Rapid visual search • Form, color, simple motion/blinking, spatial stereo depth, shading, position 12987621909023748594329087065483940560248595437289009890509874632234 Web3D Symposium 2005

  21. Web3D Symposium 2005

  22. Fundamental Data Types • Spatial / perceptual data: geometry, colors, textures, lighting • Abstract data / world & object attributes: nominal, ordinal, quantitative • Temporal data / behaviors: states, dynamics Web3D Symposium 2005

  23. Data Type Quantitative Ordinal Nominal Graphical Representation position length angle / slope area volume color / density (Cleveland and McGill, 1980) position density color texture connection containment length angle slope area volume (Mackinlay, 1986) position color texture connection containment density shape length angle slope area volume (Mackinlay, 1986) Visual Markers Web3D Symposium 2005

  24. Web3D Symposium 2005

  25. Auditory Perception • Sound {} • AudioClip {} • MovieTexture {} • pitch • intensity • Spatialized Audio (doppler effect) • Standard formats: .wav, .midi, .mp3, mpeg-1 Web3D Symposium 2005

  26. Existential Perception What is my relation to this environment? What can I do in this world? What do my senses tell me? • Viewpoint {fieldOfView} • NavigationInfo {avatarSize, headlight, visibilityLimit, type, speed} • Timesensor {cycleInterval } Web3D Symposium 2005

  27. Environmental effects • Background {}: colors and textures give a context for the environment • TextureBackground {transparency } • Fog {type color visibilityRange} • LocalFog {} & FogCoordinate {} Web3D Symposium 2005

  28. Immersion • Luciano Web3D Symposium 2005

  29. Information-Rich Virtual Environments (IRVEs) We need to understand: • How spatial/perceptual information and abstract information can be combined and displayed • What makes the combinations effective • What makes them usable and • How users think and act when using them Web3D Symposium 2005

  30. Multimedia & Comprehension • Co-references between text and images (Chandler & Sweller, 1990; Faraday & Sutcliffe, 1997, 1998) • Task Knowledge Structure (Sutcliffe & Faraday, 1994; Sutcliffe 2003) • Meaningful Learning: troubleshooting, redesigning, deriving principles (Mayer, 2002) Web3D Symposium 2005

  31. Visualization & Annotation • Challenge to maintain perceptual fidelity • Scientific Visualization overloads color, texture to show abstract information • IRVEs attempt to maintain perceptual/spatial fidelity • Grouping, registering temporal and abstract information to referents • Visual • Interactive Web3D Symposium 2005

  32. Layout Space (Locations) The layout space of abstract information in IRVEs is described by the coordinate system it is resident in: • Object • World • User • Viewport • Display Web3D Symposium 2005

  33. Object Space Object space is relative to an object’s location in the environment (e.g. Semantic Objects). Web3D Symposium 2005

  34. World Space World space is relative to an area, region, or location in the environment. Web3D Symposium 2005

  35. User Space User space is relative to the user’s location but not their viewing angle. Web3D Symposium 2005

  36. Viewport Space Viewport space-is the image plane where HUDs or overlays may be located. Web3D Symposium 2005

  37. Display Space Display layout space where abstract visualizations are located outside the rendered view in some additional screen area. Web3D Symposium 2005

  38. Working Memory How are the products of perception linked with memory and reasoning? A number of models exist: • Baddeley’s componentized WM • Barnards’s Interacting Cognitive Subsystems Web3D Symposium 2005

  39. Central Executive Visuospatial sketchpad Episodic buffer Phonological Loop Fluid system Visual semantics Episodic LTM Language Crystallized system Working Memory (Baddeley, 2003) Web3D Symposium 2005

  40. Visuospatial WM • Capacity 3-5 ‘items’ • Functional units & chunking • Objects & features (Vogel et al, 2001) • Visual indices & dynamic feature binding (Saiki, 2003) • Subsystems: form & color, space & movement (Logie 1995) • Relation to Central Executive (Miyake et al. 2001) Web3D Symposium 2005

  41. Model ICS: Interacting Cognitive Subsystems Visual Auditory Object Implicational Propositional Articulatory Motor … Web3D Symposium 2005

  42. IRVEs & Perception • Integrating information types via • Depth cues • Gestalt assocation cues • Recent studies examined search and comparison tasks: • the Viewport vs. Object space interfaces • The Display vs. Object space interfaces • Future work will investigate supporting pattern and trend recognition tasks Web3D Symposium 2005

  43. References Bertin, J., Semiology of Graphics. 1983, Madison, WI: University of Wisconsin Press. Baddeley, A., Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 2003. 4: p. 829-839. Barnard, P., May, J., Duke, D., and Duce, D., Systems, interactions, and macrotheory. ACM Trans. on Computer-Human Interaction, 2000. 7(2): p. 222-262. Biederman, I., and Gerhardstein, P.C., Recognizing depth-rotated objects: Evidence for 3-D viewpoint invariance. Journal of Experimental Psychology: Human Perception and Performance, 1993(19): p. 1162-1182. Card, S., J. Mackinlay, and B. Shneiderman, Information Visualization: Using Vision to Think. 1999, San Francisco: Morgan Kaufmann. Chandler, P., and Sweller, J., Cognitive Load Theory and the Format of Instruction. Cognition and Instruction, 1991. 8: p. 293-332. Cleveland, W.S., and McGill, R., Graphical perception: Theory, experimentation and application to the development of graphical methods. J. Am. Stat. Assoc, 1984. 79(September). Faraday, P., and Sutcliffe, Alistair. Designing Effective Multimedia Presentations. in CHI. 1997. Atlanta, GA: ACM. Goguen, J., Information Visualizations and Semiotic Morphisms. 2000, UCSD: http://citeseer.ist.psu.edu/goguen00information.html. Keller, P.R., Visual Cues: Practical Data Visualization. 1993, Piscataway, NJ: IEEE Computer Society Press. Logie, R.H., Visuo-spatial working memory. 1995, Hove, UK: Psychology Press. Mackinlay, J., Automating the Design of Graphical Presentations of Relational Information. ACM Trans. on Graphics, 1986. 5(2): p. 111-141. Marr, D., Vision : a computational investigation into the human representation and processing of visual information. 1982, San Francisco: W.H. Freeman. Mayer, R.E., Cognitive Theory and the Design of Multimedia Instruction: An Example of the Two-Way Street Between Cognition and Instruction. New Directions for Teaching and Learning, 2002. 89: p. 55-71. Norman, D.A., Cognitive Engineering, in User Centered System Design, D.A. Norman, and Draper, S.D., Editor. 1986, Lawrence Erlbaum Associates: Hillsdale, NJ. p. 31-61. Web3D Symposium 2005

  44. References Pickett, R.M., Grinstein, G., Levkowitz, H., Smith, S., Harnessing Preattentive Perceptual Processes in Visualization, in Perceptual Issues in Visualization, G. Grinstein, and Levkoitz, H., Editor. 1995, Springer: New York. Quinlan, P.T., and Wilton, R.N., Grouping by proximity or similarity? Competition between the Gestalt principles in vision. Perception, 1998. 27: p. 417-430. Rock, I., and Palmer, S., The legacy of Gestalt psychology. Scientific American, 1990(December): p. 48-61. Simmons, D.J., Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 2000. 4: p. 147-155. Sutcliffe, A., and Faraday, P. Designing Presentation in Multimedia Interfaces. in CHI. 1994: ACM Press. Tufte, E., The Visual Display of Quantitative Information. 1983, Cheshire, CT: Graphics Press. Tufte, E., Envisioning Information. 1990, Cheshire, CT: Graphics Press. Ware, C., Information Visualization: Perception for Design. 2000, New York, NY: Morgan Kauffman. http://webvision.med.utah.edu/VisualCortex.html Thanks to the ‘Rev.’ Bob Cripsen on his early instruction on VRML lighting! Web3D Symposium 2005

More Related