1 / 47

Interactive Texturing on Objects in Images via a Sketching Interface

Interactive Texturing on Objects in Images via a Sketching Interface. Kwanrattana Songsathaporn. The University of Tokyo. Henry Johan. Nanyang Technological University. Tomoyuki Nishita. The University of Tokyo. Overview. Background & System Overview Related Work

marin
Download Presentation

Interactive Texturing on Objects in Images via a Sketching Interface

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interactive Texturing on Objects in Images via a Sketching Interface KwanrattanaSongsathaporn The University of Tokyo Henry Johan Nanyang Technological University TomoyukiNishita The University of Tokyo

  2. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  3. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  4. Research Background Paste textures on textured area Paste textures on blank area • Paste textures on images, and maintain underlying shapes of objects • Applications in digital image/video manipulation

  5. Scope & Features(1/3) Drawing Photograph Interactive system for pasting textures Input image: Drawings and Photographs

  6. Scope & Features(2/3) http://en.wikipedia.org/wiki/Texture_synthesis (upper) Textures with pattern (lower) Texture without pattern Our system supports textures regardless of their patterns

  7. Scope & Features(3/3) Perspective Occlusion • Two features to make texturing easier • Systematic occlusion handling: create occlusion with one piece of texture • Systematic perspective handling: consistency to surrounding

  8. System Framework Specifications using interface Objects’ normal vector field construction Texture Coordinate Calculation Input image Output image Texture image User specifications Video Texture mapping is based on normal vector field=> independent of patterns

  9. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Parameterization • Results • Conclusions and Future Work

  10. Texturing in Images(1/3) (a) Default grid mesh (b) Aligned vertices to texels • Liu et al., SIGGRAPH 2004 • Users align vertices on grid mesh with texels to replace textures • Not support input without texture • A lot of user interactions

  11. Texturing in Images(2/3) Output image Input image Constructed normal field • Fang and Hart, SIGGRAPH 2004 • Synthesize texture on objects in images • Construct normal vector field with shape-from-shading • Constrained by limitation of shape-from-shading • Not interactive

  12. Texturing in Images(3/3) Designed normal vector field Paste texture Add shading • Winnemöller et al., EGSR 2009 • Use Diffusion Curves (DC) to design • Normal vector field • uv map • Construct normal vector field without shading • Require a lot of user interactions

  13. Comparison of Methods support not support with limitations

  14. Normal Construction Interface Specifications to transfer normal vectors 3D model from normal map • Wu et al., SIGGRAPH 2007 • Sketch-based interface: Shape palette • Transfer normal vectors from shape palettes • Easy-to-understand interface

  15. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  16. Two-Phase User Interface Objects’ normal vector field construction Texture Coordinate Calculation

  17. Two-Phase User Interface

  18. Sparse Normal Vector Specification Convex palette Concave palette Input image Normal vector field construction using our interface

  19. Meaning of Palettes z y x • Convex palette • Imaginary 3D image of convex palette Imagine that texturing region and shape palettes are 3D objects Monitor screen is image plane

  20. Two-Phase User Interface

  21. Split Stroke Specification (a) Specify texturing region (b),(c) Subregions • Discontinuity on the normal vector field • Occlusion and edge • Split single region into subregions • Initial normal vector field of each subregion can be specified separately

  22. Distance Between Subregions (a) split region (b) Distance between subregions = 0 (c) Distance between subregions > 0 (d) Black strip shows occluded part (e) Drag slide bar to increase distance Imaginary top view of the curtain Approximate occlusion region by specify distance between subregions > 0

  23. Two-Phase User Interface

  24. Correcting Perspective Projection • Furthest and nearest points (pink points) • Interface for specifying relative depth • Results from default perspective projection matrix • Result from corrected perspective projection matrix Specify depths at furthest and nearest point (pink points) Pink points are estimated by system

  25. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  26. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  27. Normal Vector Field Construction z y x Direction of x, y and z components

  28. Normal Vector Field Construction : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  29. Normal Vector Field Construction Aim to find this value of every pixel : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  30. Normal Vector Field Construction Sparsely specified normal vectors : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  31. Normal Vector Field Construction Minimize error from the specified value : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  32. Normal Vector Field Construction Enforce smoothness : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  33. Normal Vector Field Construction Adjacent pixels on the same subregion : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  34. Normal Vector Field Construction Minimize curvature : set of normal specified pixels : subregion which pixel i is member : adjacent pixels of i : set of all pixels in texturing region

  35. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  36. What We Know? Next, texture mapping Normal vector field of the objects (Normal vector field construction) Corrected perspective projection matrix (Relative depth specification)

  37. Texture Coordinate Calculation p: (u,v) known r: (u,v) unknown Goal: Find texture coordinate (u,v) of every pixels in texturing region Subregion-by-subregion, flood-filled manner from pinpoint

  38. As the calculation continues : position on image of adjacent pixel : position on image of pinpoint pixel Initial state Flood fill p: (u,v) known r: (u,v) unknown

  39. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  40. Results(1/2) (b) Retextured image (a) Input (c) Texture • Interaction time (mins) • Winnemölleret al. : 60 • Our system: 30

  41. Results(2/2) (a) Input (b,c) Retextured image • Interaction time (mins) • Winnemölleret al.: 30 • Our system: 16

  42. Limitations Distortion is visible in green box area • Distortion cumulates at distant area from pinpoint (red point)

  43. Overview • Background & System Overview • Related Work • Proposed User Interface for Texturing • Underlying Algorithms • Normal Vector Field Construction • Texture Coordinate Calculation • Results • Conclusions and Future Work

  44. Conclusions • Interactive system for texturing objects in images • Independent of texture patterns • Systematically handle occlusion • Systematically handle perspective • Reduce user interactions • Produce results with reasonable shorter time

  45. Future Work Photograph shows complex occlusion • Intuitive user interface that handles complex occlusion systematically • User study • How precisely could users perceive normal vectors on objects • How precisely could users specify normal vectors using our interface

  46. Thank You Q & A

  47. As the calculation continues Position of adjacent pixels Position of pinpoint p: (u,v) known r: (u,v) unknown

More Related