Signal Processing and Representation Theory

1. Signal Processingand Representation Theory Lecture 1

2. Outline: • Algebra Review • Numbers • Groups • Vector Spaces • Inner Product Spaces • Orthogonal / Unitary Operators • Representation Theory

3. Algebra Review Numbers (Reals) Real numbers, ℝ, are the set of numbers that we express in decimal notation, possibly with infinite, non-repeating, precision.

4. Algebra Review Numbers (Reals) Example: =3.141592653589793238462643383279502884197… Completeness: If a sequence of real numbers gets progressively “tighter” then it must converge to a real number. Size: The size of a real number aℝ is the square root of its square norm:

5. Algebra Review Numbers (Complexes) Complex numbers, ℂ, are the set of numbers that we express as a+ib, where a,bℝ and i= . Example: ei=cos+isin

6. Algebra Review Numbers (Complexes) Let p(x)=xn+an-1xn-1+…+a1x1+a0 be a polynomial with aiℂ. Algebraic Closure: p(x) must have a root, x0 in ℂ: p(x0)=0.

7. Algebra Review Numbers (Complexes) Conjugate: The conjugate of a complex number a+ib is: Size: The size of a real number a+ibℂ is the square root of its square norm:

8. Algebra Review Groups A group G is a set with a composition rule + that takes two elements of the set and returns another element, satisfying: • Asscociativity: (a+b)+c=a+(b+c) for all a,b,cG. • Identity: There exists an identity element 0G such that 0+a=a+0=a for all aG. • Inverse: For every aG there exists an element -aG such that a+(-a)=0. If the group satisfies a+b=b+a for all a,bG, then the group is called commutative or abelian.

9. Algebra Review Groups Examples: • The integers, under addition, are a commutative group. • The positive real numbers, under multiplication, are a commutative group. • The set of complex numbers without 0, under multiplication, are a commutative group. • Real/complex invertible matrices, under multiplication are a non-commutative group. • The rotation matrices, under multiplication, are a non-commutative group. (Except in 2D when they are commutative)

10. Algebra Review (Real) Vector Spaces A real vector space is a set of objects that can be added together and scaled by real numbers. Formally: A real vector space V is a commutative group with a scaling operator: (a,v)→av, aℝ, vV,such that: • 1v=v for all vV. • a(v+w)=av+aw for all aℝ, v,wV. • (a+b)v=av+bv for all a,bℝ, vV. • (ab)v=a(bv) for all a,bℝ, vV.

11. Algebra Review (Real) Vector Spaces Examples: • The set of n-dimensional arrays with real coefficients is a vector space. • The set of mxn matrices with real entries is a vector space. • The sets of real-valued functions defined in 1D, 2D, 3D,… are all vector spaces. • The sets of real-valued functions defined on the circle, disk, sphere, ball,… are all vector spaces. • Etc.

12. Algebra Review (Complex) Vector Spaces A complex vector space is a set of objects that can be added together and scaled by complex numbers. Formally: A complex vector space V is a commutative group with a scaling operator: (a,v)→av, aℂ, vV,such that: • 1v=v for all vV. • a(v+w)=av+aw for all aℂ, v,wV. • (a+b)v=av+bv for all a,bℂ, vV. • (ab)v=a(bv) for all a,bℂ, vV.

13. Algebra Review (Complex) Vector Spaces Examples: • The set of n-dimensional arrays with complex coefficients is a vector space. • The set of mxn matrices with complex entries is a vector space. • The sets of complex-valued functions defined in 1D, 2D, 3D,… are all vector spaces. • The sets of complex-valued functions defined on the circle, disk, sphere, ball,… are all vector spaces. • Etc.

14. Algebra Review (Real) Inner Product Spaces A real inner product space is a real vector space V with a mapping V,V→ℝ that takes a pair of vectors and returns a real number, satisfying: • u,v+w= u,v+ u,w for all u,v,wV. • αu,v=αu,v for all u,vV and all αℝ. • u,v= v,u for all u,vV. • v,v0 for all vV, and v,v=0 if and only if v=0.

15. Algebra Review (Real) Inner Product Spaces Examples: • The space of n-dimensional arrays with real coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then: v,w=v1w1+…+vnwn • If M is a symmetric matrix (M=Mt) whose eigen-values are all positive, then the space of n-dimensional arrays with real coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then: v,wM=vMwt

16. Algebra Review (Real) Inner Product Spaces Examples: • The space of mxn matrices with real coefficients is an inner product space.If M and N are two mxn matrices then: M,N=Trace(MtN)

17. Algebra Review (Real) Inner Product Spaces Examples: • The spaces of real-valued functions defined in 1D, 2D, 3D,… are real inner product space.If f and g are two functions in 1D, then: • The spaces of real-valued functions defined on the circle, disk, sphere, ball,… are real inner product spaces.If f and g are two functions defined on the circle, then:

18. Algebra Review (Complex) Inner Product Spaces A complex inner product space is a complex vector space V with a mapping V,V→ℂ that takes a pair of vectors and returns a complex number, satisfying: • u,v+w= u,v+ u,w for all u,v,wV. • αu,v=αu,v for all u,vV and all αℝ. • for all u,vV. • v,v0 for all vV, and v,v=0 if and only if v=0.

19. Algebra Review (Complex) Inner Product Spaces Examples: • The space of n-dimensional arrays with complex coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then: • If M is a conjugate symmetric matrix ( ) whose eigen-values are all positive, then the space of n-dimensional arrays with complex coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then: v,wM=vMwt

20. Algebra Review (Complex) Inner Product Spaces Examples: • The space of mxn matrices with real coefficients is an inner product space.If M and N are two mxn matrices then:

21. Algebra Review (Complex) Inner Product Spaces Examples: • The spaces of complex-valued functions defined in 1D, 2D, 3D,… are real inner product space.If f and g are two functions in 1D, then: • The spaces of real-valued functions defined on the circle, disk, sphere, ball,… are real inner product spaces.If f and g are two functions defined on the circle, then:

22. Algebra Review Inner Product Spaces If V1,V2V, then V is the direct sum of subspaces V1, V2, written V=V1V2, if: • Every vector vV can be written uniquely as:for some vectors v1V1 and v2V2.

23. Algebra Review Inner Product Spaces Example: If V is the vector space of 4-dimensional arrays, then V is the direct sum of the vector spaces V1,V2V where: • V1=(x1,x2,0,0) • V2=(0,0,x3,x4)

24. Algebra Review Orthogonal / Unitary Operators If V is a real / complex inner product space, then a linear map A:V→V is orthogonal / unitary if it preserves the inner product: v,w= Av,Aw for all v,wV.

25. Algebra Review Orthogonal / Unitary Operators Examples: • If V is the space of real, two-dimensional, vectors and A is any rotation or reflection, then A is orthogonal. A(v1) v1 v2 A(v2) A=

26. Algebra Review Orthogonal / Unitary Operators Examples: • If V is the space of real, three-dimensional, vectors and A is any rotation or reflection, then A is orthogonal. A=

27. Algebra Review Orthogonal / Unitary Operators Examples: • If V is the space of functions defined in 1D and A is any translation, then A is orthogonal. A=

28. Algebra Review Orthogonal / Unitary Operators Examples: • If V is the space of functions defined on a circle and A is any rotation or reflection, then A is orthogonal. A=

29. Algebra Review Orthogonal / Unitary Operators Examples: • If V is the space of functions defined on a sphere and A is any rotation or reflection, then A is orthogonal. A=

30. Outline: • Algebra Review • Representation Theory • Orthogonal / Unitary Representations • Irreducible Representations • Why Do We Care?

31. Representation Theory Orthogonal / Unitary Representation An orthogonal / unitary representation of a group G onto an inner product space V is a map  that sends every element of G to an orthogonal / unitary transformation, subject to the conditions: • (0)v=v, for all vV, where 0 is the identity element. • (gh)v=(g) (h)v

32. Representation Theory Orthogonal / Unitary Representation Examples: • If G is any group and V is any vector space, then:is an orthogonal / unitary representation. • If G is the group of rotations and reflections and V is any vector space, then:is an orthogonal / unitary representation.

33. Representation Theory Orthogonal / Unitary Representation Examples: • If G is the group of nxn orthogonal / unitary matrices, and V is the space of n-dimensional arrays, then:is an orthogonal / unitary representation.

34. Representation Theory Orthogonal / Unitary Representation Examples: • If G is the group of 2x2 rotation matrices, and V is the vector space of 4-dimensional real / complex arrays, then:is an orthogonal / unitary representation.

35. Representation Theory Irreducible Representations A representation , of a group G onto a vector space V is irreducible if cannot be broken up into smaller representation spaces. That is, if there exist WV such that: (G)WW Then either W=V or W=.

36. Representation Theory Irreducible Representations IfWV is a sub-representation of G, and W is the space of vectors perpendicular to W: v,w=0 for all vW and wW, then V=WW and W is also a sub-representation of V. For any gG, vW, and wW, we have: So if a representation is reducible, it can be broken up into the direct sum of two sub-representations.

37. Representation Theory Irreducible Representations Examples: • If G is any group and V is any vector space with dimension larger than one, then:is not an irreducible representation.

38. Representation Theory Irreducible Representations Examples: • If G is the group of 2x2 rotation matrices, and V is the vector space of 4-dimensional real / complex arrays, then:is not an irreducible representation since it maps the space W=(x1,x2,0,0) back into itself.

39. Representation Theory Why do we care?

40. Representation Theory Why we care In shape matching we have to deal with the fact that rotations do not change the shape of a model. =

41. Representation Theory Exhaustive Search If vM is a spherical function representing model M and vn is a spherical function representing model N, we want to find the minimum over all rotations T of the equation:

42. Representation Theory Exhaustive Search If V is the space of spherical functions then we can consider the representation of the group of rotations on this space. By decomposing V into a direct sum of its irreducible representations, we get a better framework for finding the best rotation.

43. Representation Theory Exhaustive Search (Brute Force) Suppose that {v1,…,vn} is some orthogonal basis for V, then we can express the shape descriptors in terms of this basis: vM=a1v1+…+anvn vN=b1v1+…+bnvn

44. Representation Theory Exhaustive Search (Brute Force) Then the dot-product of M and N at a rotation T is equal to:

45. Representation Theory Exhaustive Search (Brute Force) So that the nxn cross-multiplications are needed: v1 T(v1) + + v2 T(v2) + + = = T(vN) vM … … + + T(vn) vn

46. Representation Theory Exhaustive Search (w/ Rep. Theory) Now suppose that we can decompose V into a collection of one-dimensional representations. That is, there exists an orthogonal basis {w1,…,wn} of functions such that T(wi)wiℂ for all rotations T and hence: wi,T(wj)=0 for all i≠j.

47. Representation Theory Exhaustive Search (w/ Rep. Theory) Then we can express the shape descriptors in terms of this basis: vM=α1w1+…+αnwn vN=β1w1+…+βnwn

48. Representation Theory Exhaustive Search (w/ Rep. Theory) And the dot-product of M and N at a rotation T is equal to:

49. Representation Theory Exhaustive Search (w/ Rep. Theory) So that only n multiplications are needed: w1 T(w1) + + w2 T(w2) + + = = T(vN) vM … … + + T(wn) wn