1 / 26

Painting on Movable Objects

Dynamic Shader Lamps. Painting on Movable Objects. Deepak Bandyopadhyay, UNC-CH Ramesh Raskar, MERL Henry Fuchs, UNC-CH. Dynamic Shader Lamps. Introduction Related Work Implementation Results. Motivation.

xuxa
Download Presentation

Painting on Movable Objects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic Shader Lamps Painting on Movable Objects Deepak Bandyopadhyay, UNC-CH Ramesh Raskar, MERL Henry Fuchs, UNC-CH Deepak Bandyopadhyay / UNC Chapel Hill

  2. Dynamic Shader Lamps • Introduction • Related Work • Implementation • Results Deepak Bandyopadhyay / UNC Chapel Hill

  3. Motivation For some applications, it is easier to work with a physical object directly, rather than using traditional displays or see-through AR HMDs. • Reasons: Ergonomics, Passive Haptics • Example application: virtual 3D painting We show how to implement such systems. Deepak Bandyopadhyay / UNC Chapel Hill

  4. Projector1 Projector2 Overview Objects PC Tracker Deepak Bandyopadhyay / UNC Chapel Hill

  5. Video Deepak Bandyopadhyay / UNC Chapel Hill

  6. Applications • Testbed for AR applications • projectors on movable objects • AR Interaction medium • Art • Teleimmersion • Architecture • annotation • visualization • Others, see paper Deepak Bandyopadhyay / UNC Chapel Hill

  7. Dynamic Shader Lamps • Introduction • Related Work • Implementation • Results Deepak Bandyopadhyay / UNC Chapel Hill

  8. Shader Lamps Basic Idea Raskar et al 1, 2001 1 Ramesh Raskar, Greg Welch, Kok-Lim Low and Deepak Bandyopadhyay. “Shader Lamps : Animating Real Objectswith Image-based Illumination”, Eurographics Rendering Workshop (EGRW 2001), London. Deepak Bandyopadhyay / UNC Chapel Hill

  9. Shader Lamps Basic Idea Rearranging terms in optical path Deepak Bandyopadhyay / UNC Chapel Hill

  10. L(x,) = F(x, , i) Li(x, i) di k(x) cos(p ) d(x)2 d ( x )2 k ( x ) cos( p ) Ip (x, p) L’(x,)= L ( x, ) k (x)>0 Ip (x, p) = Radiance Adjustment Projected Real Desired Virtual Deepak Bandyopadhyay / UNC Chapel Hill

  11. Image based Illumination • Basic Idea • Render images and project on objects • Multiple projectors • View and object dependent color Deepak Bandyopadhyay / UNC Chapel Hill

  12. Related Work : 3D Painting • Hanrahan & Haeberli, 1990 2 • Painting on 3D models • Screen & mouse interface • Agrawala & Beers, 1995 3 • Painting Scanned Surfaces • Object is static • Tracked pointer moved on its surface • Separate display 2 Pat Hanrahan and Paul Haeberli, “Direct WYSIWYG Painting and Texturing on 3D Shapes”. SIGGRAPH’90. 3 Maneesh Agrawala, Andrew C. Beers and Marc Levoy, “3D Painting on Scanned Surfaces”, Symposium on Interactive 3D Graphics (I3D’95). Deepak Bandyopadhyay / UNC Chapel Hill

  13. Related Work : 3D Painting • Gregory, Ehmann & Lin, 1999 4 • inTouch haptic painting • Modeling, painting, force feedback • No real object; purely virtual • Keefe, Feliz et al, 2001 5 • CavePainting • Physical interface objects • Display is purely virtual 4 Arthur Gregory, Stephen Ehmann and Ming C. Lin, “inTouch : Interactive Multiresolution Modeling and 3D Painting with a Haptic Interface”, Virtual Reality (VR’99). 5 Daniel F. Keefe, Daniel A. Feliz, Tomer Moscovitch, David Laidlaw and Joseph LaViola, Jr., “CavePainting : a Fully Immersive 3D Artistic Medium and Interactive Experience”, I3D 2001. Deepak Bandyopadhyay / UNC Chapel Hill

  14. Dynamic Shader Lamps • Introduction • Related Work • Implementation • Results Deepak Bandyopadhyay / UNC Chapel Hill

  15. Projector Calibration • Calibrate each projector (5 minutes) • Match 3D points in world space to 2D pixels in projector space. • Solve for 3x4 projection matrix • ignore radial distortion • minimize least-square error (standard CV method) • split into intrinsic (projection) and extrinsic matrix • Render 3D points in the world with good registration 2D cross hair Deepak Bandyopadhyay / UNC Chapel Hill

  16. Model Generation/Authoring • Need: vertices, faces, normals, texture coordinates and color/material scheme per object to be painted • Acquire points using magnetic stylus • Faces, textures by hand / surface reconstruction • Texture coordinates non-repeating, fit geometry • Future: automate with modeling tools. Deepak Bandyopadhyay / UNC Chapel Hill

  17. Rendering Tracked Object • Algorithm each-frame: Tracker gives 6DOF reading of sensors. For view i (projector i) do Load projection and model view matrix i For object j do Apply world to sensor j matrix • Draw object in sensor coordinates • Interact with object Deepak Bandyopadhyay / UNC Chapel Hill

  18. 3D painting interface • Objects hand-held or set on table. • Tracked stylus with spherical tip • facilitates contact painting • Projected touch palette, modeled as a static object with behavior: • choose contact, spray or texture paint • choose brush color Deepak Bandyopadhyay / UNC Chapel Hill

  19. R r Painting objects • Determine object intersections (point-polygon) • Find set of polygons to modify • Color blending using brush function 6 • Contact paint uses a function on surface of object B = 1-(r/R)2, r<R • Spray paint uses a similar brush function in 3D • Color (or texture) blending: Cfinal = Cinit.(1-B) + Cbrush.B 6 From Arthur Gregory’s inTouch system Deepak Bandyopadhyay / UNC Chapel Hill

  20. Polygon in 3D 2D texture map q concurrent (u,v) v (x,y,z) u p Look up brush fn Update color Painting objects • Triangle rasterization w/ texture map modification 6 • Scan 2D and 3D simultaneously. 6 From Arthur Gregory’s inTouch system Deepak Bandyopadhyay / UNC Chapel Hill

  21. Dynamic Shader Lamps • Introduction • Related Work • Implementation • Results Deepak Bandyopadhyay / UNC Chapel Hill

  22. Performance / numbers ? • 60fps rendering 3 objects, 500+ polygons • ~100 texture maps, mostly 128x128 • Running on SGI InfiniteReality2 or 450MHz PC with multi-monitor graphics card • Not optimized for larger data sets • 100-110ms latency (high). • Roughly estimated • 1cm of distance lag when moved at 10cm/s • Later, precisely measured Deepak Bandyopadhyay / UNC Chapel Hill

  23. Issues and Future Work • Tracker Limitations • Range (Polhemus Fastrak, 50-75 cm) • use wide-area tracker • Technology (ferrous objects distort magnetic field) • Display limitations • Dynamic Intensity Blending • Limited dynamic range of projector • Occlusion, overlap, shadows • Scaling it up • Room sized environments, multiple projectors Deepak Bandyopadhyay / UNC Chapel Hill

  24. Conclusions • Dynamic Shader Lamps • A set of techniques for interactively changing appearance attributes of neutrally colored physical objects, static or moving • Prototype implementation • Painting on multiple real objects • display devices - projectors • 3D display surfaces (real objects) • 3D input devices - tracker & stylus • Extensions - how to generalize/scale. Deepak Bandyopadhyay / UNC Chapel Hill

  25. Acknowledgements • ISAR reviewers • Miriam Mintzer Fuchs, age 10 • People at UNC (paper, video & demo) • Herman Towles, Greg Welch, Anselmo Lastra, Russ Taylor, Ming Lin, Scott Russell • Andrei State, Jim Mahaney, Kurtis Keller, John Thomas, David Harrison, Arthur Gregory • Kok-Lim Low, Wei-Chao Chen, Ben Lok, Kenny Hoff, Dave McAllister, Miguel Otaduy, Gentaro Hirota • STC Tracked Shader Lamps, funding Deepak Bandyopadhyay / UNC Chapel Hill

  26. Painting on Movable Objects Dynamic Shader Lamps Images Deepak Bandyopadhyay, Ramesh Raskar, Henry Fuchs http://www.cs.unc.edu/~debug/papers/DSLpaint/

More Related