Properties Of the Quadratic Performance Surface

1. Properties Of the Quadratic Performance Surface Lecture Three

2. The Quadratic Performance Surface • From derivation of Weiner Hopf; Quadratic Performance Surface is expressed by: •  = min + (W – W*)TR (W – W*) • Quadratic Performance Surface is a function of autocorrelation matrix (R) of input vector • The Eigenanalysis of the autocorrelation matrix gives information about the characteristics of the Performance Surface

3. Introduction • From the standpoint of Engineering applications eigenvalue problem are among the most important problems in connection with matrices. • For a linear discrete time and continuous linear systems matrix A completely determine system stability. • The eigenvectors of A forms a very convenient choice for the eigenvectors . • Eigenvectors may be used to uncouple the state equations and determine convenient way for the system analysis

4. Definition • Let the domain and range of a linear transformation be D(A) and R(A) within the vector space X. • Those vectors xi and scalars λi which satisfy the condition A(xi)= λixi are called as eigenvectors and eigenvectors respectively. • Case for xiis excluded.

5. Eigenvalues • Necessary condition for the existence of solution to set of n homogeneous equations is that rank(A- Iλi )<n • Which results • For Linear transformation this becomes

6. If there are p<n distinct roots • The integer mi is called the algebraic multiplicity of λi • Equation has to be solved for the roots to get the Eigenvalues

7. Eigenvectors and Eigenvalues Problems • Membrane Stretch problem • An elastic membrane is stretched such that a point P goes over into point Q • Find the principle directions of stretch

9. Governing equations for the stretch is • Eigenvalues for transformation matrix are 2 and 8

11. Original and Stretched Membrane

12. Thus the eigenvectors specify dimensions along which the output is directed for the specific value of the input vector. • Output is just the integral multiple of input at these values • The constant factor is called as the eigenvalue • The output is eigenvector corresponding to that eigenvalue

13. The Quadratic Performance Surface • Using Eigen analysis we can get the idea of basis of the performance surfaces on which it is defined • Hence analysis of the performance surface is simplified

14. Normal Form of the Input Correlation Matrix • The Eigenvalues of the input autocorrelation matrix R is defined as • R Qn = n Qn • Where Qn is the nth eigenvector corresponding to n nth eigenvalue • The eigenvalues are computed from the following characteristic equation • det[R - I] = 0

15. Normal Form of the Input Correlation Matrix • Eigenvector form the basis vectors for the input autocorrelation matrix R. • And Eigenvalues are the weights of the vectors. • We can write

16. Normal Form of the Input Correlation Matrix • Therefore we can also write • RQ = Q or R = QQ-1 • This is the normal form of R • Where Q = [Q0 Q1 … QL], is the eigenvector matrix • And  is a diagonal matrix with eigenvalue as the diagonal entries and is called ‘eigenvalue matrix’

17. Properties of the Eigenvalues and Eigenvectors • As R is a symmetric matrix, the eigenvectors corresponding to distinct eigenvalues are mutually orthogonal. • Since R is real, all eigenvalues must be real and greater than or equal to zero • The eigenvector matrix Q can be normalized such that QQT = I

18. Geometrical Significance of the Eigenvectors and Eigenvalues • The eigenvectors and the eigenvalues are related to certain properties of the error surface. • We know that the error performance surface form a hyperparabolic surface in a space of N dimensions for N-1 weights.

19. Geometrical Significance of the Eigenvectors and Eigenvalues MSE w1 w0 The hyperparabolic surface of three dimensions for 2 weights

20. Geometrical Significance of the Eigenvectors and Eigenvalues • If we cut the paraboloid with planes parallel to the w0w1-plane, we obtain concentric ellipses corresponding to different values of mean square error.

21. Geometrical Significance of the Eigenvectors and Eigenvalues v1 v’1 v’0 v0 w1 w0 Ellipses with different color shades corresponding to different values of mean square error

22. Geometrical Significance of the Eigenvectors and Eigenvalues • From mean square error expression, equation for ellipses it can be written as • WTRW – 2PTW = constant • Using the Alternate expression for gradient we can also write it as • VTRV = constant • The general expression for the ellipses in function form can be written as • F(V) = VTRV

23. Geometrical Significance of the Eigenvectors and Eigenvalues • A vector normal to the ellipses can be obtained by taking gradient of F • The principal axis of the ellipse pass through the origin and therefore is of form V

24. Geometrical Significance of the Eigenvectors and Eigenvalues • Also the principal axis is normal to the ellipses F(V),therefore • 2RV’ = V’ • [R – (/2)I] V’ = 0 • Thus V’ is the principle axis and also the eigenvector of the matrix R.

25. Geometrical Significance of the Eigenvectors and Eigenvalues • The eigenvectors of the input correlation matrix define the principle axes of the error surface

26. Geometrical Significance of the Eigenvectors and Eigenvalues • Take the expression for the mean square error •  = min + VTR V • Where V = (W – W*) • Replace R by its normal form • R = QQ-1 • We have •  = min + V’T V’

27. Geometrical Significance of the Eigenvectors and Eigenvalues • The gradient of above expression would be •  = 2 V’ • = 2[0v’0 1v’1 … Lv’L] • To summarize • V = (W – W*) can be considered as a translation to a new axis • V’ = QTV is the transformation to the principal coordinate system

28. Geometrical Significance of the Eigenvectors and Eigenvalues • The gradient of  along any principal axis is given as • Thus the eigenvalues of the input correlation matrix R give the second derivative of the error surface, , with respect to the principal axes of 

29. Assignment • All exercise Questions from Chapter 2 • MATLAB questions due in next lab Tuesday