370 likes | 409 Views
Evolution of Graphics Hardware: 40 years since Sutherland’s HMD. Henry FUCHS University of North Carolina at Chapel Hill Graphics Hardware 2008 Sarajevo. Acknowledgement Assistance with preparation of this talk. Hideyuki Tamura, Canon Greg Welch, UNC Mark Mine, Disney
E N D
Evolution of Graphics Hardware:40 years since Sutherland’s HMD • Henry FUCHS • University of North Carolina at Chapel Hill • Graphics Hardware 2008 Sarajevo
AcknowledgementAssistance with preparation of this talk • Hideyuki Tamura, Canon • Greg Welch, UNC • Mark Mine, Disney • Steve Molnar, NVIDIA • Adam Lake, Intel
Evolution of Graphics H/W: from what point ? • Ivan Sutherland’s 1963 PhD Sketchpad • Beginning of graphics, but no new hardware • Sutherland’s 1965 “Ultimate Display” idea • A vision, but no specific HW specified • Sutherland’s 1968 3D HMD System • Complete working system, needed all specialized HW
Ivan E. SutherlandA Head-Mounted Three-Dimensional Display,1968 Fall Joint Computer Conference • Implemented all the components of a real-time interactive 3D graphics system: • Display device: stereo, optical see-through • Image generation: Real-time 3D perspective • Head tracking: Real-time 6 DOF • 3D model creation • Hand-tracking & interaction: Real-time 3 DOF (’69 ?)
Procedure for Examining Evolution of Graphics HW • For each subsystem: display, image generation, tracking.. • 1968: what was implemented, state of the art • ~1988: sample state of the art • 2008: current example subsystem • Assessment: • The dream in 1968/1970 • Grade for where we are now: A, B, C, D
Display device on ’68 HMD • Optical see-through head-mounted display
HMD ca. 1985: VPL head-mount • Closed display, not see-through • LCD from SONY pocket TV • wide angle optics (LEEP) • (no distortion correction)
HMD ca. 1995: video see-through HMD (UNC) • Need for video, rather than optical, see-through
Current HM Displays • closed, total VR • Fakespace wide5 • Video see-through: • Canon / Mixed Reality Labs VH2002 • InnerOptic (based on UNC design)
Assessment on HM Displays • The dream at 1968 / 1970: Displays built in to our ordinary eyeglasses; see virtual objects throughout the eyeglass lenses (not a ‘virtual inset’) • Grade for where we are now: C • significant progress • not even close to adequate
Image generation:Sutherland’s 1968 system • Real time • Line drawing • No hidden line elimination • Heroic work: everything built out of gate-level chips • 3D transforms • Clipping divider • Graphics pipeline
Image generation: ca 1988 • Real-time full-color raster image generation • Rapid, consistent progress throughout the 1980s and 1990s • 100s of polygons /sec in 1980 • Million polygons / sec early 1990s • UNC Pixel-Planes 5 at siggraph 1991: 2M polygons/sec
Image generation now • Sophisticated shading effects and millions of polygons /sec • Commodity graphics chips & boards • Rapid, continuing progress • Image generation processors becoming ubiquitous –migrating even to mobile devices
Image Generation Assessment • Dream of 1968/1970: Realistic image generation • Sutherland 1965 Ultimate display: can’t distinguish between virtual and real objects • Grade for where we are now: • A+ • Are we finished? • No! illuminate globally; integrate physics sim. with graphics; integrate image capture with image display,.. • Feels great for the field to be doing so well.
Head tracking: Sutherland’s 1968 system • Two tracking systems implemented • ultrasound tracking with multiple emitters and receivers • mechanical tracker
Head tracking (ultrasonic)Sutherland’s 1968 system • Ultrasonic tracking • 3 transmitters on head-mount • 4 receivers hanging from ceiling • Measure phase changes • Problem with ambiguity of number of cycles of u/s signals • Problem exacerbated by heating system
Head tracking (mechanical)Sutherland’s 1968 system • Vertical pivot in ceiling • Universal joint on top • Universal joint on bottom • Shaft slides in and out • “Sword of Damocles” • Heavy and uncomfortable • Works
Head Tracking ~1988: Polhemus Magnetic Tracking • Developed for head tracking in cockpit • Very close range • Not for walking across a room • Severe warping of space with metal, with other magnetic fields. Can be reduced with calibration
Large area trackers: ~ 1991 UNC • 10x12 ft space demonstrated at Siggraph 1991 • Multiple optical sensors on head, LEDs in ceiling tiles • Lateral effect photo-diodes, much faster than imagers • One LED lighted at a time • Predict most-useful LED to light each time slot
Head tracking Now • Multiple good solutions • Optical • Magnetic • Inertial / optical combination • Limited to instrumented area
Head tracking assessment • Dream of 1968 / 1970: Go anywhere, unencumbered • Grade for where we are now: • B • Choice of solutions on the market if willing to live with restrictions • Degraded performance if lose line of sight contact (optical trackers) or near ferrous/metal objects (magnetic trackers) • Can’t go outside of instrumented area, nor outdoors. • Steady progress
Interaction / Hand tracking in Sutherland’s HMD • Mechanical design with three reels of fishing line mounted from ceiling • All lines connected to top of (camera) hand-grip • Amount of line reeled out determines 3D location of hand-grip • Problem of interference between head and hand trackers
Interaction / Hand tracking ca.1988 • Polhemus magnetic tracker (time-shared with a target for head-tracker) • VPL DataGlove sometimes added to hand-tracker • Severe warping of tracked space if user moves more than a few steps
Interaction / Hand tracking now • Multiple technologies available • Accuracy & speed may not be adequate for certain applications • UNC AR assistance for surgical needle guidance • Track ultrasound transducer • Track needle
Assessment of Hand tracking • Dream in 1968/ 1970: unencumbered tracking of hand • Grade for where we are now: • Incomplete • Haven’t had sufficient number of real applications to judge --- rest of system hasn’t been ready • using Fred Brooks’ denition of “What’s real about Virtual Reality?” • Someone other than develops pay to use the system
3D model creation • Sutherland’s 1968 system: trivially simple
3D model creation ca. 1988 • Manual model creation increasingly expensive / time-consuming with increasing capability of image generation systems • Feasible if 3D model needed to be created as part of the application itself: CAD/CAM, selected medical applications • Automatic scanning for selected objects: Cyberware 3D scanners
3D Model Creation Now • Growing list of commercial and research capabilities • Computer vision: multiple images / video to 3D world models • Laser scanners • Motion capture systems
Assessment of Model Creation • Dream in 1968 / 1970: • Sutherland: Building model may be as time-consuming as building the real thing • Others: There’s got to be an automatic way • Grade for where we are now • C+ • There are many tools and systems, but often still takes much tedious, manual effort • Compare how much more work to digitizing this room and its contents than to rendering it
Assessment Summary • Display: • perennially, next model will be really good C • Image generation: • unalloyed triumph! (but don’t let it go to your head) A+ • Head tracking: • effective but in limited, prepared spaces B • Interaction / Hand tracking: • awaiting more real applications Incomplete • Model generation: • frustratingly time-consuming C+
Opportunities: Look at larger picture than a single module • Head-mounted display design combined with image generation • Fakespace Wide5: hardware geometric distortion correction • Mark Mine (Disney) implementing more sophisticated version in GPU • Image generation combined with head-tracking • Model creation combined with image generation • System problem spread across multiple modules: • latency
Sutherland’s vision of the HMD as the penultimate display still inspires
In conclusion:Don’t constrain your dreams;there is so much cool graphics HW to invent • Stereo and autostereo displays • Multi-projector / camera systems • Vision capture for image-based rendering • Haptics in the small and large • Cell phones w/graphics & vision: magic lenses in the real world • Telepresence systems (could do an entire hour on this) • Finally– Alan KayThe best way to predict the future is to invent itTHE END 35