Shape Analysis and Retrieval

Shape Analysis and Retrieval Statistical Shape Descriptors Notes courtesy of Funk et al., SIGGRAPH 2004

Outline: • Shape Descriptors • Statistical Shape Descriptors • Singular Value Decomposition (SVD)

Shape Matching General approach:Define a function that takes in two models and returns a measure of their proximity. D , D , M1 M2 M1 M3 M1 is closer to M2 than it is to M3

D , D , Shape Descriptors Shape Descriptor:A structured abstraction of a 3D model that is well suited to the challenges of shape matching 3D Models Descriptors

Matching with Descriptors Preprocessing • Compute database descriptors Run-Time 3D Query ShapeDescriptor BestMatches 3D Database

Matching with Descriptors Preprocessing • Compute database descriptors Run-Time • Compute query descriptor 3D Query ShapeDescriptor BestMatches 3D Database

Matching with Descriptors Preprocessing • Compute database descriptors Run-Time • Compute query descriptor • Compare query descriptor to database descriptors 3D Query ShapeDescriptor BestMatches 3D Database

Matching with Descriptors Preprocessing • Compute database descriptors Run-Time • Compute query descriptor • Compare query descriptor to database descriptors • Return best Match(es) 3D Query ShapeDescriptor BestMatches 3D Database

Shape Matching Challenge Need shape descriptor that is: • Concise to store • Quick to compute • Efficient to match • Discriminating 3D Query ShapeDescriptor BestMatches 3D Database

Shape Matching Challenge Need shape descriptor that is: • Concise to store • Quick to compute • Efficient to match • Discriminating • Invariant to transformations • Invariant to deformations • Insensitive to noise • Insensitive to topology • Robust to degeneracies Different Transformations(translation, scale, rotation, mirror)

Shape Matching Challenge Need shape descriptor that is: • Concise to store • Quick to compute • Efficient to match • Discriminating • Invariant to transformations • Invariant to deformations • Insensitive to noise • Insensitive to topology • Robust to degeneracies Different Articulated Poses

Shape Matching Challenge Need shape descriptor that is: • Concise to store • Quick to compute • Efficient to match • Discriminating • Invariant to transformations • Invariant to deformations • Insensitive to noise • Insensitive to topology • Robust to degeneracies Scanned Surface Image courtesy ofRamamoorthi et al.

Shape Matching Challenge Need shape descriptor that is: • Concise to store • Quick to compute • Efficient to match • Discriminating • Invariant to transformations • Invariant to deformations • Insensitive to noise • Insensitive to topology • Robust to degeneracies Different Genus Different Tessellations Images courtesy of Viewpoint & Stanford

Shape Matching Challenge Need shape descriptor that is: • Concise to store • Quick to compute • Efficient to match • Discriminating • Invariant to transformations • Invariant to deformations • Insensitive to noise • Insensitive to topology • Robust to degeneracies No Bottom! &*Q?@#A%! Images courtesy of Utah & De Espona

Statistical Shape Descriptors Challenge:Want a simple shape descriptor that is easy to compare and gives a continuous measure of the similarity between two models. Solution:Represent each model by a vector and define the distance between models as the distance between corresponding vectors.

Statistical Shape Descriptors Properties: • Structured representation • Easy to compare • Generalizes the matching problem Models represented as points in a fixed dimensional vector space

Statistical Shape Descriptors General Approaches: • Retrieval • Clustering • Compression • Hierarchicalrepresentation Models represented as points in a fixed dimensional vector space

Complexity of Retrieval Given a query: • Compute the distance to each database model • Sort the database models by proximity • Return the closest matches ~ M1 M1 ~ ~ M2 M2 M1 D(Q,Mi) Q 3D Query ~ M2 ~ Mk Mk Best Match(es) Database Models Sorted Models

Complexity of Retrieval If there are k models in the database and each model is represented by an n-dimensional vector: • Computing the distance to each database model: • O(k n) time • Sort the database models by proximity: • O(k logk) time If n is large, retrieval will be prohibitively slow.

Algebra Definition: Given a vector space V and a subspace WV, the projection onto W, written W, is the map that sends vV to the nearest vector in W. If {w1,…,wm} is an orthonormal basis for W, then:

Tensor Algebra Definition: The inner product of two n-dimensional vectors v={v1,…,vn} and w={w1,…,wn}, written v,w, is the scalar value defined by:

Tensor Algebra Definition: The outer product of two n-dimensional vectors v={v1,…,vn} and w={w1,…,wn}, written vw, is the matrix defined by:

Tensor Algebra Definition: The transpose of an mxn matrix M, written Mt, is the nxm matrix with: Property: For any two vectors v and w, the transpose has the property:

SVD Compression Key Idea:Given a collection of vectors in n-dimensional space, find a good m-dimensional subspace (m<<n) in which to represent the vectors.

SVD Compression Specifically:If P={p1,…,pk} is the initial n-dimensional point set, and {w1,…,wm} is an orthonormal basis for the m-dimensional subspace, we will compress the point set by sending:

SVD Compression Challenge:To find the m-dimensional subspace that will best capture the initial point information.

Variance of the Point Set Given a collection of points P={p1,…,pk}, in an n-dimensional vector space, determine how the vectors are distributedacross different directions. p1 pi p2 pk

Variance of the Point Set Define the VarP as the function:giving the variance of thepoint set P in direction v(assume |v|=1). p1 pi p2 pk

Variance of the Point Set More generally, for a subspace WV, define the variance of P in the subspace W as:If {w1,…,wm} is an orthonormal basis for W, then:

Variance of the Point Set Example: The variance in the direction v1is large, while the variance inthe direction v2 is small.  If we want to compressdown to one dimension, weshould project the points onto v1 p1 pi v1 p2 v2 pk

Covariance Matrix Definition: The covariance matrixMP, of a point set P={p1,…,pk} is the symmetric matrix which is the sum of the outer products of the pi:

Covariance Matrix Theorem: The variance of the point set P in a direction v is equal to:

Covariance Matrix Theorem: The variance of the point set P in a direction v is equal to: Proof:

Singular Value Decomposition Theorem: Every symmetric matrix M can be written out as the product:where O is a rotation/reflection matrix (OOt=Id) and D is a diagonal matrix with the property:

Singular Value Decomposition Implication: Given a point set P, we can compute the covariance matrix of the point set, MP, and express the matrix in terms of its SVD factorization:where {v1,…,vn} is an orthonormal basis and i is the variance of the point set in direction vi.

Singular Value Decomposition Compression: The vector subspace spanned by {v1,…,vm} is the vector sub-space that maximizes the variance in the initial point set P. If m is too small, then too much information is discarded and there will be a loss in retrieval accuracy.

Singular Value Decomposition Hierarchical Matching: First coarsely compare the query to database vectors. • If {query is coarsely similar to target} • Refine the comparison • Else • Do not refine O(k n) matching becomes O(k m) with m<<n and no loss of retrieval accuracy.

Singular Value Decomposition Hierarchical Matching: SVD expresses the initial vectors in terms of the eigenbasis:Because there is more variance in v1 than in v2, more variance in v2 than in v3, etc. this gives a hierarchical representation of the data so that coarse comparisons can be performed by comparing only the first m coefficients.

Efficient to match? Preprocessing: • Compute SVD factorization • Transform database descriptors Run-Time: • Transform Query Query SVD SVD

Efficient to match? • Low resolution sort Database Query 0.040 0.052 0.103 0.430 0.661 Distance to Query

Efficient to match? • Update closest matches • Resort Database Query 0.229 0.052 0.103 0.430 0.661 Distance to Query

Shape Analysis and Retrieval