1 / 57

Interactive Point-based Modeling of Complex Objects from Images

Interactive Point-based Modeling of Complex Objects from Images. Pierre Poulin ( a,b ) Marc Stamminger ( a,c ) François Duranleau ( b ) Marie-Claude Frasson ( a ) George Drettakis ( a ) ( a ) REVES, INRIA Sophia Antipolis ( b ) DIRO, Université de Montréal ( c ) University of Erlangen.

lise
Download Presentation

Interactive Point-based Modeling of Complex Objects from Images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Interactive Point-based Modeling of Complex Objects from Images Pierre Poulin (a,b) Marc Stamminger (a,c) François Duranleau (b) Marie-Claude Frasson (a) George Drettakis (a) (a) REVES, INRIA Sophia Antipolis (b) DIRO, Université de Montréal (c) University of Erlangen

  2. Modeling Complex Objects

  3. Modeling Complex Objects • High visual complexity • Time consuming • Algorithms for specialized objects • e.g., plants, mountains, etc. • Adaptive rendering • Many applications need such objects

  4. Key Observations • Extracting complex models from photos is a very powerful approach • Point-based representation is very effective for complex models • Efficient display and storage • User interaction is beneficial when extracting quality models • Specify where details are needed • Resolve some ambiguities

  5. Image-based Point Modeling • Images are very flexible • Reality-based (photos) • Acquisition is easy

  6. Image-based Point Modeling • Points are very flexible • Fast rendering (hardware support) • Adaptive rendering for interactive display Stamminger

  7. Image-based Point Modeling • Points are very flexible • Hierarchical organization and levels of detail Q-splat

  8. Image-based Point Modeling • Points are very flexible • Visual quality • Many recent advances Deussen

  9. Images Constraints Reconstruction Process 3D Model Image Automatic Reconstruction

  10. Images Constraints Reconstruction Process 3D Model Image Interactive Reconstruction new images requantize recalibrate User

  11. Images Constraints Reconstruction Process 3D Model Image Interactive Reconstruction color comparisons plausibility threshold new depth maps zone of interest User

  12. Images Constraints Reconstruction Process 3D Model Image Interactive Reconstruction User revalidate the points request more points decimate the points jitter the points sample with patterns hole filling

  13. Images Constraints Reconstruction Process 3D Model Image Interactive Reconstruction User undo changes remove points add polygons

  14. Interactive Reconstruction • Interactive display • 6 M points/sec. on a PIII 1GHz with GeForce3 • Efficient reconstruction algorithm • Test more than 1K points/sec. • Simple and intuitive controls • Direct interaction with the points

  15. Computer Vision Contributions • 3D scanners • Structured light • Stereo – N-views • Shape-from-X • Volumetric

  16. Volumetric Reconstruction • Voxel coloring and Space carving • If a voxel is impossible, carved out of object • Silhouettes, transparency, shading • Photo-consistency Kutulakos Seitz

  17. Image-based Polygon Modeling • Academic: Façade, Rekon, Reality • Industry: RealViz, Canoma, Photomodeler Façade

  18. Image-based Polygon Modeling • Small polygonal scene (30-100 polygons) • Extracted textures and illumination Boivin

  19. Input Images (4/14)

  20. Input Images • Digital camera: Canon EOS-DS30 • 1080x720 and 2166x1440 • Fixed aperture and shutter speed • Try not to change zoom • OpenGL and ray traced test scenes

  21. Camera Calibration

  22. Camera Calibration • ImageModeler from RealViz • Fiduciary marks placed around the object • Interactive system • Intrinsic and extrinsic camera parameters

  23. 3D Zone of Interest

  24. Initial Random Points

  25. Initial Random Points • Generated randomly within the envelope • More specific patterns discussed later • Projection of a point in each photo • Gather colors

  26. Color Comparison • Euclidean distance • RGB, CIE xy, CIE Luv, CIE Lab • Speed vs. accuracy • Color quantized images • Precomputed (ppmquantall or more sophisticated) • Quantization only on projected zone of interest • 32 to 128 colors • Reduce shading variations • Efficient test for color equality

  27. Dominant Color Plausibility with visibility A: 100% 100% B: 50% 33% C: 25%

  28. Random Points with Depth Maps

  29. Depth Maps • Computed from the current set of points • Updated on user demand • With depth maps, can raise the plausibility threshold • Generate more points within the object • Re-evaluation of previously generated points

  30. Clean-up Points

  31. Clean-up Points • In general • Increase color threshold and re-evaluate • With good depth maps • Project in each image • Reject if point visible and color too different

  32. Generate More Points

  33. Generate More Points • Randomly • Stratified sampling based on voxels • Point decimation based on voxels

  34. Guide the Points

  35. Guide the Points • Smaller 3D sphere of interest • Generate more points • Eliminate all points • 3D flood fill for branching patterns • Patterns for planar surfaces • Patterns for boundary surfaces

  36. Filling with no Leaves

  37. Filling with Leaves

  38. Jitter the Points

  39. Reprojection

  40. Stepping through it again

  41. Results

  42. Synthetic Fruit Bowl color points reprojection ray tracing

  43. Toy Soldier color points reprojection color points

  44. Snack

  45. Snack

  46. Ficcus

  47. Conclusions • Point-based reconstruction of complex objects from images • Tight integration • 3D color point representation • User-driven and/or automatic reconstructions • Interactive display • Flexible to integrate most advances in computer vision

  48. Findings • First steps are encouraging, but objects are still of limited realism • Information in photos is inspiring, but also difficult to analyse correctly • How many things in a pixel? • How many pixels and colors for an object?

  49. Future Work • Video sequences • High dynamic range photos • Shadows and shading in color comparison • Extraction of limited BRDFs • 3D texture synthesis of materials

More Related