510 likes | 516 Views
Explore the Princeton Shape Benchmark, a comprehensive evaluation of 3D model shape descriptors. Discover the best shape retrieval techniques using various descriptors, evaluation tools, and visualization software.
E N D
The Princeton Shape BenchmarkPhilip Shilane, Patrick Min, Michael Kazhdan, and Thomas Funkhouser
Shape Retrieval Problem 3D Model ShapeDescriptor BestMatches Model Database
Example Shape Descriptors • D2 Shape Distributions • Extended Gaussian Image • Shape Histograms • Spherical Extent Function • Spherical Harmonic Descriptor • Light Field Descriptor • etc.
Example Shape Descriptors • D2 Shape Distributions • Extended Gaussian Image • Shape Histograms • Spherical Extent Function • Spherical Harmonic Descriptor • Light Field Descriptor • etc. How do we know which is best?
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results Query
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results Query
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results Query
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results
Typical Retrieval Experiment • Create a database of 3D models • Group the models into classes • For each model: • Rank other models by similarity • Measure how many models in the same class appear near the top of the ranked list • Present average results
Outline • Introduction • Related work • Princeton Shape Benchmark • Comparison of 12 descriptors • Evaluation techniques • Results • Conclusion
Aerodynamic Typical Shape Databases
Letter ‘C’ Typical Shape Databases
Typical Shape Databases 153 dining chairs 25 living room chairs 16 beds 12 dining tables 8 chests 28 bottles 39 vases 36 end tables
Goal: Benchmark for 3D Shape Retrieval • Large number of classified models • Wide variety of class types • Not too many or too few models in each class • Standardized evaluation tools • Ability to investigate properties of descriptors • Freely available to researchers
Princeton Shape Benchmark • Large shape database • 6,670 models • 1,814 classified models, 161 classes • Separate training and test sets • Standardized suite of tests • Multiple classifications • Targeted sets of queries • Standardized evaluation tools • Visualization software • Quantitative metrics
51 potted plants 33 faces 15 desk chairs 22 dining chairs 100 humans 28 biplanes 14 flying birds 11 ships Princeton Shape Benchmark
Outline • Introduction • Related work • Princeton Shape Benchmark • Comparison of 12 descriptors • Evaluation techniques • Results • Conclusion
Comparison of Shape Descriptors • Shape Histograms (Shells) • Shape Histograms (Sectors) • Shape Histograms (SecShells) • D2 Shape Distributions • Extended Gaussian Image (EGI) • Complex Extended Gaussian Image (CEGI) • Spherical Extent Function (EXT) • Radialized Spherical Extent Function (REXT) • Voxel • Gaussian Euclidean Distance Transform (GEDT) • Spherical Harmonic Descriptor (SHD) • Light Field Descriptor (LFD)
Evaluation Tools Visualization tools • Precision/recall plot • Best matches • Distance image • Tier image Quantitative metrics • Nearest neighbor • First and Second tier • E-Measure • Discounted Cumulative Gain (DCG)
Evaluation Tools Visualization tools • Precision/recall plot • Best matches • Distance image • Tier image Quantitative metrics • Nearest neighbor • First and Second tier • E-Measure • Discounted Cumulative Gain (DCG)
Evaluation Tools Query Correct class Visualization tools • Precision/recall plot • Best matches • Distance image • Tier image Quantitative metrics • Nearest neighbor • First and Second tier • E-Measure • Discounted Cumulative Gain (DCG) Wrong class
Evaluation Tools Visualization tools • Precision/recall plot • Best matches • Distance image • Tier image Quantitative metrics • Nearest neighbor • First and Second tier • E-Measure • Discounted Cumulative Gain (DCG)
Evaluation Tools Visualization tools • Precision/recall plot • Best matches • Distance image • Tier image Quantitative metrics • Nearest neighbor • First and Second tier • E-Measure • Discounted Cumulative Gain (DCG)
Evaluation Tools Visualization tools • Precision/recall plot • Best matches • Distance image • Tier image Quantitative metrics • Nearest neighbor • First and Second tier • E-Measure • Discounted Cumulative Gain (DCG) Dining ChairDesk Chair
Rectangular table Function vs. Shape • Functional at the top levels of the hierarchy, shape based at the lower levels root Man-made Natural Vehicle Furniture Table Chair Round table
Base Classification (92 classes) Man-made Furniture Table Round table
Coarse Classification (44 classes) Man-made Furniture Table Round table
Coarser Classification (6 classes) Man-made Furniture Table Round table
Coarsest Classification (2 classes) Man-made Furniture Table Round table
Granularity Comparison Base(92) Man-made vs. Natural (2)
Conclusion • Methodology to compare shape descriptors • Vary classifications • Query lists targeted at specific properties • Unexpected results • EGI: good at discriminating man-made vs. natural objects, though poor at fine-grained distinctions • LFD: good overall performance across tests • Freely available Princeton Shape Benchmark • 1,814 classified polygonal models • Source code for evaluation tools
Future Work • Multi-classifiers • Evaluate statistical significance of results • Application of techniques to other domains • Text retrieval • Image retrieval • Protein classification
Acknowledgements David Bengali partitioned thousands of models. Ming Ouhyoung and his students provided the light field descriptor. Dejan Vranic provided the CCCC and MPEG-7 databases. Viewpoint Data Labs donated the Viewpoint database. Remco Veltkamp and Hans Tangelder provided the Utrecht database. Funding: The National Science Foundation grants CCR-0093343 and 11S-0121446.
The End http://shape.cs.princeton.edu/benchmark