1 / 21

Low-Level Plumbing for Media Integration

Low-Level Plumbing for Media Integration Turner Whitted Microsoft Research Outline Part I: Implementation chronology Then, Now, Then again … aka Wheel Of Reincarnation Part II: Architectural musing Perceptual/content requirements Data paths, data types Not about programming models

issac
Download Presentation

Low-Level Plumbing for Media Integration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Low-Level PlumbingforMedia Integration Turner Whitted Microsoft Research

  2. Outline • Part I: Implementation chronology • Then, Now, Then again … • aka Wheel Of Reincarnation • Part II: Architectural musing • Perceptual/content requirements • Data paths, data types • Not about programming models

  3. Starting points – E&S Frame Buffer • NO fixed function units • mCode for basic logic • mController programmable by designer only • Treated as peripheral Ref: Kajiya, J.T., Sutherland, I.E., and Cheadle, E.C., "A Random-Access Video Frame Buffer," Proceedings of the Conference on Computer Graphics, Pattern Recognition, and Data Structure, UCLA Extension, Los Angeles, California, May 14-16, 1975

  4. Starting points – Ikonas RDS3000 • Single, 32-bit wide datapath based on 2901 bit-sliced DSP • Programmed in C (Gary’s Ikonas Assemler) • NO fixed function units • Bound [later memory mapped] to single application Ref: N. England, A graphics system architecture for interactive application-specific display functions, IEEE CGA, pp. 60-70, Jan 1986.

  5. Starting points – Pixar CHAP • SIMD processors • Loops, conditional execution • Focus on parallel programming issues Ref: Adam Levinthal and Thomas Porter, “Chap – A SIMD Graphics Processor,” Proceedings of SIGGRAPH 84, (18) 3, July 1984, pp. 77 – 82.

  6. Host Bus Example: AT&T Pixel Machine One or, optionally two, 9-element 16, 20, 32, 40, or 64 geometry pipelines Pixel Nodes “The” pipeline • [Mostly] Fixed structure, programmable nodes • Vector graphics legacy

  7. Structural evolution • Texture engine with remnants of line drawing DNA

  8. Performance • Ikonas RDS3000 (1980) • 20 MB/s processor to memory • 20 MIPS equiv. • Pixar CHAP (1984) • 240 MB/s processor to memory (P-bus) • 64 MIPS peak • ATI 9700 (2002) • 20800 MB/s chip to memory • 8400 MIPS

  9. API abstractions (OpenGL, DX) • Fixed structure pipeline accessed through API • API tracks hardware through several generations while maintaining consistency • Integrated with mainstream computing • Easy to program • Single largest key to commercial success of graphics systems and applications

  10. Complete the circle • FPGA+memory • NO fixed function • Programmable in C (Verilog actually) • Good for simple prototyping • No API • No device drivers • No SDK

  11. Part II: Going forward … • Motivation • We’ve tacked every imaginable feature onto what was initially a line drawing pipeline • It’s time to start over • The window of opportunity is wide open

  12. Alternatives Feed eyeballs with Integrated media Real-time photorealism Parallel execution for scientific applications Parallel execution for interactive applications Going forward …with graphics processors 3D Raster Graphics

  13. Integrated media(partial illustration) Concentric mosaics Image centric Geometry centric Sprites with depth View-dependent texture View-dependent geometry Light field Lumigraph Fixed geometry Polygon rendering + texture mapping Interpolation Warping

  14. Geometry Image completely regular sampling 3D geometry geometry image257 x 257; 12 bits/channel Ref: X. Gu, S. Gortler, H. Hoppe, “Geometry images,” ACM Transactions on Graphics 21(3): 355-361 (2002)

  15. From first principles: function U-VWL* Content Line drawing Display Processor Display Device Video Still photos Animated 3D shapes Text *Ultra-Vast Wasteland

  16. 3D text experiment • Extend IBR/volume rendering to text • Superior image reconstruction • Higher image quality than mip-mapped texture Olynyk, Mitchell, Snyder, MSR

  17. General purpose front end Fixed function back end Generic physical blocks Read Mapper Cache Common Memory Recon- Write Struction/ Filtering Cache Ref: T. Whitted, “Overview of IBR:Hardware and Software Issues,” ICIP 2000.

  18. Don’t count on quantum GPUs soon – stick to CMOS digital logic Count on CAD more than feature size Heat is the enemy The economy of commodity DRAM is hard to beat But there is huge performance pressure on DRAM Designers are restricted only by a lack of experience CMOS digital circuitry Commodity DRAM From first principles: implementation Content

  19. Design challenges • Essence of the problem • We don’t have a function to implement • We must design specifically for unknown methods • Brute force is prohibited • Feeds and speeds …What do we know? • Regulated by content • But we rarely turn content conventions into quantitative measures • Limited by perception • Which we don’t fully understand • Function of representation • If we don’t know the representation, we don’t know the flows • In my group’s research we limit flow to HW “sanity” and then work backwards

  20. Summary • Graphics hardware has nearly completed the circuit back to its starting point • Flexible, powerful, programmable • Media processing requirements extend beyond the classic 3D pipeline • Unusual window of opportunity • to match architecture with a broader range of applications and content

More Related