1 / 61

3D and Immersive Interfaces

3D and Immersive Interfaces. 3D Interfaces – Shneiderman. An “extreme” interpretation of dm in which interfaces approach richness of real 3D world is wrong … to paraphrase Shneiderman In fact 3D interfaces are most often not right, even for tasks for which they may seem appropriate

sierra
Download Presentation

3D and Immersive Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D and Immersive Interfaces

  2. 3D Interfaces – Shneiderman • An “extreme” interpretation of dm in which interfaces approach richness of real 3D world is wrong … to paraphrase Shneiderman • In fact 3D interfaces are most often not right, even for tasks for which they may seem appropriate • E.g., air traffic control • When used 3D interfaces have a daunting set of challenges • E.g., occlusion, navigation in 3 space • E.g., “cone tree” rep. of file structure • So …

  3. 3D Interfaces – Shneiderman • An “extreme” interpretation of dm in which interfaces approach richness of real 3D world is wrong • In fact 3D interfaces are most often not right, even for tasks for which they may seem appropriate • E.g., air traffic control • When used 3D interfaces have a daunting set of challenges • E.g., occlusion, navigation in 3 space • E.g., “cone tree” rep. of file structure • So, …

  4. Introduction • The “best” interfaces, and all systems, typically find their task utility through engagement (etc.) appropriate for the task • This idea is at the core of arguments for the use of direct manipulation interfaces • All of the following are interrelated: • Immersion, engagement, presence, virtual reality • 3D display and interaction devices • In field of CS and HCI: “spatial interfaces” and “immersive interfaces” • Also, will introduce the idea of presence

  5. Examples of Immersive Interfaces Surround screen projection Tiled display wall Spherical projection Head mounted display

  6. Immersion, “Virtual Reality”, and Virtual Environments • Immersive interfaces • High sensory immersion – visual, auditory, haptic, proprioceptive • “Virtual reality”, or, virtual environments • “Virtual reality is a technology that is used to generate a simulated environment in digital form... Using the equipment, users are immersed in a totally virtual world.” • Working definition – an immersive interactive system • In context of “virtual reality”, immersion usually = spatial immersion • Note: “Immersion” (and engagement and presence) is a continuum • Text ... Visual and 3d .. Stereo ... HMD… “jacked in” • Cyberspace • Term coined by Gibson in Neuromancer • … and in the 21st century, the Matrix

  7. Immersion and Virtual Reality • “The mind has a strong desire to believe that the world it perceives is real” – Jaron Lanier, among others • For example, “illusion” (perception) of depth (for spatial immersion) • Stereo parallax • Head motion parallax • Object motion parallax • Texture scale • Interaction: grab and move an object • Proprioceptive cues: • when you reach out and see a hand where you believe your hand to be, you accept the hand as your own • Often you will accept what you see as “real” even if graphics poor • Constellation of cues

  8. Components of Spatial ImmersionCutting, 1996 • Perception of 3 dims strongest element of spatial immersion • Perception of 3d from depth cues • See figure • Other elements • Integration important • Visual display types • Stereoscopic display • Head position sensing • Hand-position sensing • Force feedback • Sound input and output • Other sensations

  9. Presence “The Aesthetic Impression of 3D Space” • Sense of presence • Vividly 3d • Actually present in the world • Sense of being there • Holodeck … • Presence has to do as much with engagement, as visual information • E.g., one can be “in the world”, when reading • Here, one sees, or visualizes, the world • 3D depth cues are those elements that enhance feeling of 3 (vs. 2) dimensions in a display, • From occlusion to stereoscopic display

  10. Presence “The Aesthetic Impression of 3D Space” • Immersive interfaces • term used to describe interfaces/devices which lead toward immersion (sense of presence, engagement) in the virtual environment presented on the display • Virtual reality interfaces • term used similarly to immersive interfaces • Degree of immersion • conventional desktop screen • fishtank virtual reality (semi-immersive workbench) • immersive virtual reality • augmented reality with video or optical blending • … number of cues …

  11. Pictorial Depth CuesWhere does perception of three dimensions come from? • 3D depth cues • Static / pictorial vs. dynamic • Monocular vs. binocular • Oculomotor • Static monocular cues • Occlusion • Relative size • Linear perspective • Texture gradient • Aerial perspective • Shading • Relative height • Courses in computer graphics and visualization provide detail

  12. History - Sutherland’s Sketchpad“In the beginning …” • Ivan Sutherland • “Pioneer” of … lots of things • Visualization • Graphics • Interaction • Still around • Evans and Sutherland graphics • First truly interactive graphics system, Sketchpad • A fairly sophisticated “paint” (or drawing) program • MIT, Ivan Sutherland’s 1963 Ph.D. thesis • “Sketchpad, A Man-Machine Graphical Communication System” • Available: www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf • Video: www.youtube.com/watch?v=mOZqRJzE8xg • Among most important works in computer science Ivan Sutherland using Sketchpad in 1963 CRT monitor, light pen and function-key panel

  13. Ivan Sutherland’s Sketchpad, 1963and “the ultimate display” • Regarded as the first to implement much of what called “hci”, “visualization”, “immersion”, and “virtual reality” (not to mention cg) • Some quotes: • ….. If the task of the display is to serve as a looking-glass into the mathematical wonderland constructed in computer memory, it should serve as many senses as possible. • ….. By working with such displays of mathematical phenomena we can learn to know them as well as we know our own natural world. Such knowledge is the major promiseof computer displays. • ….. The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.

  14. Sensorama, 1965Less Profound maybe, but Fun – • Morton Heilig • cinematographer / director of documentaries • Motorcycle simulator - all senses • visual (city scenes) • sound (engine, city sounds) • vibration (engine) • smell (exhaust, food) • Not a big commercial success, but “immersive” from Virtual Reality Technology, Burdea & Coiffet

  15. USAF Super Cockpit, 1985 • Wright Patterson Air Force Base • Visual, auditory, tactile • Head, eye, speech, and hand input • Designed to deal with problem of pilot information overload • Flight controls and tasks too complicated • Research only • big system, not safe for ejecting

  16. Immersive and 3D Interfaces • Teleoperation • Virtual and augmented reality • Immersion and VR – contribution of components … • Survey of 3D displays • Surround screen displays - CAVE • Input devices - Data glove • Data walls • Workbenches • Hemispherical display • Head-mounted displays • Arm-mounted displays • Virtual retinal display • Autostereoscopic displays

  17. Immersive and 3D Interfaces • Degree of immersion • conventional desktop screen • Some 3D displays: • Data walls • Workbenches • Hemispherical display • Head-mounted displays • Arm-mounted displays • Surround screen displays - CAVE • Virtual retinal display • Autostereoscopic displays • But first, … about stereoscopic display • A “compelling” 3D depth cue • Is that part of human computer interaction • Yes

  18. 3D Interfaces, Stereopsis“Discovery” of Stereopsis, 1838 • Charles Wheatstone • Prolific scientist, Wheatstone bridge • “… the mind perceives an object of three-dimensions by means of the two dissimilar pictures projected by it on the two retinae…”, 1838 • Contributions to the Physiology of Vision.—Part the First. On some remarkable, and hitherto unobserved, Phenomena of Binocular Vision. • "Philosophical Transactions" of the Royal Society of London, Vol. 128, pp. 371 – 394, 1838. • http://www.stereoscopy.com/library/wheatstone-paper1838.html What the left and right eye see (retinal image)

  19. Getting Different Images to Each Eye Wheatstone’s Stereoscope • “The stereoscope is represented by figs. 8. and 9; the former being a front view, and the latter a plan of the instrument. A A' are two plane mirrors, about four inches square, inserted in frames, …” • Below, modern mirror stereoscope using computer monitors

  20. Getting Different Images to Each Eyethese days • Commodity televisions • 120 hz display rate • Alternately, left then right eye image • Active lcd glasses, alternately block left eye, then right eye image • lcd lens is alternately transparent, then opague • 60 hz left eye, • 60 hz right eye • Theaters • Polarized glasses

  21. Getting Different Images to Each Eye Sutherland’s 1960’s equipment • “Ultimate display”, 1965 • Sword of Damocles – 1st HMD • Actual camera-like shutters • Actual camera-like metal shutters

  22. Visual Displays for VEs(Bowman) • Types: • Standard monitor (mono/stereo) • Head-mounted/head-referenced • Projected (usually stereo) • single-screen • multiple, surrounding screens • Retinal display • Volumetric displays • Characteristics of visual displays • Field of regard (FOR), field of view (FOV) • Brightness, contrast ratio • Resolution (two definitions) • Screen geometry • Light transfer • Refresh rate • Ergonomics

  23. Remote (or tele-) Operation • Combines: • direct manipulation in personal computers • process control in complex environments • Physical operation is remote • Submarines, rovers, operating rooms • Complicating factors in architecture of remote environments: • Time delays • transmission delays • operation delays • Incomplete feedback • Feedback from multiple sources • Unanticipated interferences

  24. Virtual and Augmented Reality • Augmented reality shows real world with an overlay of additional overlay • Knowlton (1975) • Partially-silvered mirror over keyboard • Programmable labels • Tactile feedback

  25. Augmented Reality, 1 • Enables users to see real world with an overlay of additional interaction • Situational awareness • Typically, add text+images to real world • See through glasses • Very sensitive to head tracking, when used

  26. Augmented Reality, 2 • Enables users to see real world with an overlay of additional interaction • Situational awareness • See through glasses • Typically, add text+images to real world • Very sensitive to head tracking, when used

  27. Augmented Reality, 3 • Heads up displays • Wearable displays - military applications

  28. Augmented Reality, 4 • Heads up displays • Wearable displays - general use - Microvision

  29. Surround-screen displays • Pro • less obtrusive headgear • multi-user? • better stereo • Con • occlusion problem • missing sides

  30. Immersive 3D DisplaysSurround Screen Systems • Essentially same multi-surface display and interaction paradigm used today as 1992 • Orders of magnitude less hardware and software cost

  31. Immersive 3D Displays Surround Screen Systems • Essentially same multi-surface display and interaction paradigm used today as 1992 • Orders of magnitude less hardware and software cost Surface & Illumination1 Surface & Illuminationn User Tracking Software - modeling, … Computer(s) . . . Cruz-Neira et al., 1992

  32. Surround screen displays – CAVE, 1 • A room with walls and/or floor formed by rear projection screens • Head tracking • Stereo • Light scattering problems • Visual immersion • Field of view is 100% possible, ~200 degrees

  33. Surround screen displays – CAVE, 2 • Typical size: 10’ x 10’ x 10’ room • 2 or 3 walls are rear projection screens • Floor is projected from above • User is • tracked • He/she also wears stereo shutter goggles… • Uses “wand” to manipulate • Projects 3D scenes for viewer’s point of view on walls • Walls vanish, user perceives full 3D scene • So, view is only correct for that viewer • Cost is fairly high

  34. UTPA Immersive Systems Lab~Spring, 2014 27’ 13’ Front Projection Screen Computers Proj. TV CAVE 21’ storage Development Physiological Measurement Equipment Proj. Proj. ~8’ Proj. 6’ Security area

  35. 3D Input Devices, 1 • 3D input is hard • Electromagnetic trackers • 6 DOF (position and orientation in space) • can be attached to any head, hands, joints • must deal with noise, calibration • Polhemus tracker • Optical trackers • photogrammetric technique: space-resection by collinearity • no EM interference to worry about • self-calibration • UNC’s Highball

  36. 3D Input Devices, 2 • Gloves – “gesture recognition” • attach electromagnetic tracker to the hand • “breaks” fiberoptics • Pinch gloves • contact between digits is a “pinch” gesture • Mouselike • 6 DOF • Logicad Magellan controller

  37. Tactile Feedback • Another avenue

  38. “Data Walls” • Very widespread use • Literature of practical use • Can use commodity projector • E.g., with 27 as 3x9, with 1kx1k each gives • 9,000 x 3,000

  39. Workbenches, 1 • “One wall of CAVE” – rear stereo projection – fishtank view • UNC NanoManipulator • Below with force feedback to “feel” carbon nanotubes with Atomic Force Microscope

  40. Workbenches, 2 • Immersadesk is best know

  41. Tabletop displays • Pros • direct manipulation • “god’s-eye” view • change orientation • Cons • cancellation problem • small FOR • Virtual retinal display (VRD) • HIT lab / Microvision • image scanned • directly onto retina great potential

  42. Hemispherical display • As with mirror stereoscope, high resolution possible

  43. Head Mounted Displays, 1 • HMDs • Relatively high field of view (fov) • 90o direct FOV, 140o corneal FOV • LEEP Optics (1975) • Large Expanse, Extra Perspective (LEEP) • Eric Howlett (Pop-Optix Labs) • Originally for 3D still photo viewing • Reported great realism for still images • Lenses correct for camera distortion • Display optics matched to camera optics • Often uncorrected distortion for CG images • And, more current technology:

  44. Head Mounted Displays, 2 • HMDs • Relatively high field of view (fov), ~ 140 x 60 • NASA Ames HMD (1981-1984) • McGreevy and Humphries • First implemented immersive HMDs • LCD “Watchman” displays • NASA Ames VIEW or VIVID (1985) • Virtual Interface Environment Workstation • Polhemus tracker, LEEP-based HMD, 3D audio, Crystal River’s Convolvotron, Gesture recognition w/ VPL DataGlove, BOOM-mounted CRT (Sterling Software), Remote Camera (Fake Space)

  45. HMDs now

  46. http://www.vrealities.com/proviews035.html DISPLAY Image Source Type: Single Full-color AMOLED (800x3 pixels) x 600 linesBrightness: 25 foot lamberts Contrast: >50:1 OPTICAL Field of View: 32° Horizontal x24° Vertical (40° Diagonal) @ SVGA resolution (800x600)26° Horiztonal x 19° Vertical (32° Diagonal) @ VGA resolution (640x480)Transmission: Non see-throughOptics: Plastic aspherical lensEye Relief: Eyeglasses compatible, >25 mmExit Pupil: Non-pupil forming MECHANICAL Mounting: Standard: Clip on to Mich helmet* (Display module); Clip on to belt (Disply Controller)Weight: Display Module: 67 grams (w/out mount); Helmet Mount: 79 grams; Display Controller: 200 grams DIMENSIONS Display Module (less mount): Height: 0.7"; Width: 1.7"; Depth: 1.2">br> Electronics Box: Height: 3.0"; Width: 2.8"; Depth: 1.2" Cable: Length: 1.2 meters INTERFACE Plugable Display Module: Display Module includes 51" cable (Interface cable available separately) with water-resistant connectorDisplay Controller: Lemo Connector ELECTRONIC Analog Video Rates: 800 x600 (SVGA 56 Hz to 85 Hz), 640x480 (VGA 60 Hz to 85 Hz)Analog Video Inputs: NTSC/RS-170 (A); CVBS and Y/C; R,G,B,HS,VSPower: System (VIN 6-24 VDC, 2.5 W nominal)Controls: RS-232 brightness and channel selection DESIGNED TO MEET THE FOLLOWING PARAMETERS Temperature: Operating: -32° to +55°C; Storage: -32°C to +71°CHumidity: Six 48-hour cycles, 20°C to 55°C, 95% RHSalt Fog: Four 24-hour cycles (two wet, two dry)Vibration: Random vibration, 6 axis, 5 Hz to 2500Hz, up to 40 gsImmersion: Immersion in 1 meter of water for 2 hours VR Price: $9,995.00 Add a Head Tracker! CLICK HERE Integrated Microphone with Voice Recognition Software Add $125.00 Wanting another payment option? Questions on products? Need help? CLICK HERE

  47. http://www.cwonline.com/store/view_product.asp?Product=1151 • $799.00In StockThe new i-glasses PC models utilize LCOS micro-displays from Brillian Corporation with 800 x 600 resolution. These displays have been designed to achieve a much greater range of contrast and brightness capability. This results in an image substantially more vibrant and vivid. The new models will also reach their optimal operating temperature much faster and offer a very consistent contrast and color fidelity over the entire display. With built on integrated headphones. The Ultimate Portable PC Video System. Available in 3D model too! • SpecificationsResolution: 800x600 • Field of View: 26 Degrees Diagonal Image • Size: 76" at 13 Color Depth: 24 Bit Input • IPD Adjustments: None Required • Focus: 13 TBR • Eye Relief: 25mm • Exit Pupil: 17mmH x 6mmV • Convergence: 710", 100% Overlap, TBR • VGA / SVGA / XVGA Input: Scaled to SVGA (800 x 600) • Refresh Rate: Flicker Free 100Hz • Audio: Full Stereo Weight: 7 Ounces • Adjustments: Adjusts to fit all individuals • Control Features: On / Off, Volume Control • Power Supply: 100-240V AC Power Cube • Audio Connector: 1/8" Power: Barrel Connector

  48. http://www.vrealities.com/addvisor150.html • The AddVisor 150 personal display system is designed for the high-end professional market for Head-Mounted Displays with AR and VR capability. The AddVisor 150 is a transparent high-resolution Head-Mounted Display. The image is generated on two independent full colour 1280x1024 pixels (SXGA) Liquid Crystal on Silicon microdisplays. A light weight solution for the most demanding of applications the AddVisor 150 is recognized for excellent image quality, high brightness and contrast. The AddVisor 150 is ideal for direct view mono- and stereo (3D) viewing. • The image can be shown superimposed on the environment with up to 35 % see-through or fully immersed. The AddVisor 150 is designed for a 46 degree diagonal 100% overlap field of view. A 50% overlap can also be used, giving a 54 degree horizontal or 60 degree diagonal field of view. It features a patented optical design that combines a wide field of view with high transparency see-through. Using a patent pending head fitting system with easy adjustments, low weight and eyeglass compatibility, the AddVisor 150 provides hours of easy and comfortable viewing. • Specifications: • Resolution: 1280 x 1024 - SXGAField of View: 60 Degrees DiagonalImage Size: 76" at 13' Color Depth: 24 Bit InputIPD Adjustments: None RequiredEye Relief: Eyeglass compatibleConvergence: 7'10", 100% Overlap, TBRAudio: OptionalWeight: < 1 kgAdjusts to Fit all IndividualsControl Features: On / Off, Volume ControlPower Supply: 100-240V AC Power CubePower: Barrel Connector • Call or E-mail for Price

  49. Stereoscopic Viewing(Bowman) • Stereopsis • Static, binocular cue • Each eye gets a slightly different image • Only effective within a few feet of viewer • Many implementation schemes • Motion parallax • Dynamic, monocular cue • Near objects move faster than far objects • Generally more important than stereo! • Oculomotor cues • Based on information from muscles in the Eye • Accommodation: lens shape (depth of focus) • (Con)vergence: gaze direction

  50. Fyi - Stereoscopic Viewing(Bowman) • Accommodation-convergence mismatch • Standard stereo displays confuse the brain based on oculomotor cues • Only “true 3D” displays can provide these

More Related