Texture Compression for Large Real Environments

1. Texture Compression for Large Real Environments Yizhou Yu Computer Science Division University of California at Berkeley

2. Varying Scene Configurations • Segment geometry into individual objects • Related work • [ Hoffman & Jain 87 ], [ Besl & Jain 88 ], [ Newman Flynn & Jain 93 ], [ Leonardis, Gupta & Bajcsy 95 ] Original Configuration Novel Configuration

3. Input Multiple range scans of a scene Multiple photographs of the same scene Output Geometric meshes of each object in the scene Registered texture maps for objects Framework photograph 3D mesh synthetic image range scan

4. Overview Range Images Point Cloud Point Groups Meshes Registration Segmentation Reconstruction Pose Estimation Radiance Images Texture Maps Objects

5. Image Segmentation as Graph Partitioning Build a weighted graph G=(V,E) from image V: image pixels E: connections between pairs of nearby pixels Partition graph so that similarity within group is large and similarity between groups is small -- Normalized Cuts Approximate solution from a generalized eigenvalue problem [Shi&Malik 97]

6. Segmentation Results

7. Image

8. Aligning Photographs with Laser Scans • Pose estimation using calibration targets • 3 rotation and 3 translation parameters • Combinatorial search • 4 correspondences each image 3D Targets

9. Camera Pose Results • Accuracy: consistently within 2 pixels • Correctness: correct pose for 58 out of 62 images

10. Reconstructed Mesh with Camera Poses and Calibration Targets The “crust” algorithm, nearest-neighbors & quadric error metric

11. Models of Individual Objects

12. Texture Map Synthesis I • Conventional Texture-Mapping with Texture Coordinates • Create a triangular texture patch for each triangle • The texture patch is a weighted average of the image patches from multiple photographs • Pixels that are close to image boundaries or viewed from a grazing angle obtain smaller weights Photograph 3D Triangle Texture Map

13. Texture Map Synthesis II • Allocate space for texture patches from texture maps • Generalization of memory allocation to 2D • Quantize edge length to a power of 2 • Sort texture patches into decreasing order and use First-Fit strategy to allocate space First-Fit

14. A Texture Map Packed with Triangular Texture Patches

15. Texture-Mapping and Object Manipulation Original Configuration Novel Configuration

16. Texture Map Compression I • The size of each texture patch is determined by the amount of color variations on its corresponding triangles in photographs. • An edge detector (the derivative of the Gaussian) is used as a metric for variations.

17. Results on Edge Detection

18. Texture Map Compression II • Reuse texture patches • Map the same patch to multiple 3D triangles with similar color variations • K-mean clustering to generate texture patch representatives • Larger penalty along triange edges to reduce Mach Band effect • Binary search to find the number of clusters 3D Triangles Texture Map

19. Synthetic Images with Compressed and Uncompressed Texture Maps Compressed 5 texture maps Uncompressed 20 texture maps 20 texture maps 5 texture maps

20. Video