Signal Processing and Representation Theory

Signal Processingand Representation Theory Lecture 2

Outline: • Review • Invariance • Schur’s Lemma • Fourier Decomposition

Representation Theory Review An orthogonal / unitary representation of a group G onto an inner product space V is a map  that sends every element of G to an orthogonal / unitary transformation, subject to the conditions: • (0)v=v, for all vV, where 0 is the identity element. • (gh)v=(g) (h)v

Representation Theory Review If we are given a representation of a group G onto a vector space V, then WV is a sub-representation if: (g)wW for every gG and every wW. A representation of a group G onto V is irreducible if the only sub-representations are WV are W=V or W=.

Representation Theory Review Example: • If G is the group of 2x2 rotation matrices, and V is the vector space of 4-dimensional real / complex arrays, then:is not an irreducible representation since it maps the space W=(x1,x2,0,0) back into itself.

Representation Theory Review Given a representation  of a group G onto a vector space V, for any two elements v,wV, we can define the correlation function: Corr(g,v,w)=v, (g)w Giving the dot-product of v with the transformations of w.

Representation Theory Review (Why We Care) Given a representation  of a group G onto a vector space V, if we can express V as the direct sum of irreducible representations: V=V1…Vn then: • Alignment can be solved more efficiently by reducing the number of multiplications in the computation of the correlation. • We can obtain (robust) transformation-invariant representations.

Representation Theory Review (Why We Care) Correlation: v1 T(v1) w1 T(w1) + + + + v2 T(v2) w2 T(w2) + + + + … … … … + + + + T(vn) T(wn) vn wn

Representation Theory Motivation If vM is a spherical function representing model M and vn is a spherical function representing model N, we want to define a map Ψthat takes a spherical function and return a rotation invariant array of values: • Ψ(vM)=Ψ(T(vM)) for all rotations T and all shape descriptors vM. • ||Ψ(vM)-Ψ(vN)|| ||vM-vN|| for all shape descriptors vM and vN.

Representation Theory More Generally Given a representation  of a group G onto a vector space V, we want to define a map Ψthat takes a vector vV and returns a G-invariant array of values: • Ψ(v)=Ψ((g)v) for all vV and all gG. • ||Ψ(v)-Ψ(w)|| ||v-w|| for all v,wV.

Representation Theory Invariance Approach: Given a representation  of a group G onto a vector space V, map each vector vV to its norm: Ψ(v)=||v|| • Since the representation is unitary, ||(g)v||=||v|| for all vV and all gG. Thus, Ψ(v)=Ψ((g)v) and the map Ψ is invariant to the action of G. • Since the difference between the size of two vectors is never bigger than the distance between the vectors, we have ||Ψ(v)-Ψ(w)||||v-w|| for all v,wV.

Representation Theory Invariance If V is an inner product space, v,wV, we know that: w v-w ║||v||-||w||║ v

Representation Theory Invariance Example: Consider the representation of the group of 2x2 rotation matrices onto the vector space of 4-dimensional arrays:Then the map:is a rotation-invariant map…

Representation Theory Invariance Example: … but so is the map: The new map is better because it gives more rotation invariant information about the initial vector.

Representation Theory Invariance Generally: Given a representation  of a group G onto a vector space V, if we can express V as the direct sum of sub-representations: V=V1…Vn then expressing a vector v as the sum v=v1+…+vn with viVi, we can define the rotation invariant mapping:

Representation Theory Invariance Generally: The finer the resolution, (i.e. the bigger n is) the more rotation invariant information is captured by the mapping: Thus, the best case is when each of the Vi is an irreducible representation.

Representation Theory Invariance Why is the mapping Ψ invariant? If v=v1+…+vn is any vector in V, with viViand gG then we write out: (g)v=w1+…+wn where wiViand we get:

Representation Theory Invariance Why is the mapping Ψ invariant? We can also write out: (g)v=(g)v1+…+(g)vn. Since the Vi are sub-representations we know that (g)viVi, giving two different expressions for (g)v as the sum of vectors in Vi: (g)v=w1+…+wn (g)v=(g)v1+…+(g)vn

Representation Theory Invariance Why is the mapping Ψ invariant? However, since V is the direct sum of the Vi: V=V1…Vn we know that any such decomposition is unique, and hence we must have: wi= (g)vi and consequently:

Representation Theory Schur’s Lemma Preliminaries: • If A is a linear map A:V→V, then the kernel of A is the subspace WV such that A(w)=0 for all wW.

Representation Theory Schur’s Lemma Preliminaries: • If A is a linear map, the characteristic polynomial of A is the polynomial: • The roots of the characteristic polynomial, the values of λ for whichPA(λ)=0, are the eigen-values of A. • If V is a complex vector space and A:V→V is a linear transformation, then A always has at least one eigen-value. (Because of the algebraic closure of ℂ.)

Representation Theory Schur’s Lemma Lemma: If G is a commutative group, and  is a representation of G onto a complex inner product space V, then if V is more than one complex dimensional, it is not irreducible. So we can break up V into a direct sum of smaller, one-dimensional representations.

Representation Theory Schur’s Lemma Proof: Suppose that V is an irreducible representation and larger than one complex-dimensional… Let hG be any element of the group. Then for every hG and every vV, we know that: (g)(h)(v)=(h)(g)(v).

Representation Theory Schur’s Lemma Proof: Since (h) is a linear operator we know that it has a complex eigen-value λ. Set A:V→V to be the linear operator: A=(h)- λI. Note that because G is commutative and diagonal matrices commute with any matrix, we have: (g)A=A(g) for all gG.

Representation Theory Schur’s Lemma Proof: A=(h)- λI Set WV to be the kernel of A. Since λ is an eigen-value of A, we know that W≠.

Representation Theory Schur’s Lemma Proof: Then since we know that: (g)A=A(g), for any wW=Kernel(A), we have: (g)(Aw)=0 A((g)w)=0. Thus, (g)wW for all gG and therefore we get a sub-representation of G on W.

Representation Theory Schur’s Lemma Proof: Two cases: • Either W≠V, in which case we did not start with an irreducible representation.  • Or, W=V, in which case the kernel of A is all of V, which implies that A=0 and hence (h)=λI.Since this must be true for all hG, this must mean that every hG, acts on V by multiplication by a complex scalar.Then any one-dimensional subspace of V is an irreducible representation. 

Algebra Review Fourier Decomposition If V is the space of functions defined on a circle and G is the group of rotations about the origin, then we have a representation of G onto V: If g is the rotation by 0 degrees, then g sends the function f() to the function f(-0). f() f(-0) 0 g=

Algebra Review Fourier Decomposition Since the group of 2D rotations is commutative, by Schur’s lemma we know that there exists one-dimensional sub-representations ViV such that V=V1…Vn …

Algebra Review Fourier Decomposition Or in other words, there exist orthogonal, complex-valued, functions {w1(),…,wn(),…} such that for any rotation gG, we have: (g)wi() =λi(g)wi() with λi(g)ℂ.

Representation Theory Fourier Decomposition The wk are precisely the functions: wk()=eik And a rotation by 0 degrees acts on wk() by sending:

Representation Theory Fourier Decomposition If f() is a function defined on a circle, we can express the function f in terms of its Fourier decomposition:with akℂ.

Representation Theory Fourier Decomposition Invariance / Power Spectrum / Fourier Descriptors: If f() is a function defined on a circle, expressed in terms of its Fourier decomposition:then the collection of norms:is rotation invariant.

Fourier Descriptors CircularFunction

Fourier Descriptors … = + + + + CircularFunction Cosine/Sine Decomposition

Fourier Descriptors … = + + + + CircularFunction = Constant Frequency Decomposition

Fourier Descriptors … = + + + + + CircularFunction = Constant 1st Order Frequency Decomposition

Fourier Descriptors … = + + + + + CircularFunction = + Constant 1st Order 2nd Order Frequency Decomposition

Fourier Descriptors … = + + + + + CircularFunction … = + + Constant 1st Order 2nd Order 3rd Order Frequency Decomposition

Fourier Descriptors Amplitudes invariantto rotation … = + + + + CircularFunction … = + + Constant 1st Order 2nd Order 3rd Order Frequency Decomposition

Representation Theory Fourier Decomposition Correlation: If f() and h() are function defined on a circle, expressed in terms of their Fourier decomposition:

Representation Theory Fourier Decomposition Correlation: then the correlation of f with g at a rotation  is: Convolution in the spatial domain is equivalent to multiplication in the frequency domain.

Representation Theory Fourier Decomposition Two (circular) n-dimensional arrays can be correlated by computing the Fourier decompositions, multiplying the frequency terms, and computing the inverse Fourier decomposition. • Computing the forward transforms: O(n log n) • Multiplying Fourier coefficients: O(n) • Computing the inverse transform: O(n log n) Total running time for correlation: O(n log n)

Representation Theory How do we get the Fourier decomposition?

Representation Theory Fourier Decomposition Preliminaries: If f is a function defined in 2D, we can get a function on the unit circle by looking at the restriction of f to points with norm 1.

Representation Theory Fourier Decomposition Preliminaries: A polynomial p(x,y) is homogenous of degree d if it is the sum of monomials of degree d: p(x,y)=ad xd+ad-1xd-1y+…+a1 xyd-1+a0 yd monomials of degree d

Representation Theory Fourier Decomposition Preliminaries: If we let Pd(x,y) be the set of homogenous polynomials of degree d, then Pd(x,y) is a vector-space of dimension d+1:

Signal Processing and Representation Theory