Introduction to Mathematical Methods in Computational Bioscience(CB)

1. Introduction to Mathematical Methods in Computational Bioscience(CB) Chapter 2Linear Algebra Department of Mathematics & Computer Science

2. Linear Algebra • Linear Algebra (LA) has widespread applications throughout mathematics • LA concepts such as bases of vector spaces and e-values and e-vectors arise in the traditional introductory ordinary differential equations (ode) course in the study of linear d.e’s. and systems of linear d.e’s. Department of Mathematics & Computer Science

3. A few notations • Vector in Linear Algebra is different from Vector in Biology • Domain in mathematics is different from domain in biology Department of Mathematics & Computer Science

4. 1. Linear Equations • Let’s consider the following problem An osmorphilic yeast colony is grown in the medium of varying sugar concentration. If the concentration of the sugar is 1000 gl-1, the growth rate is 0 h-1; if the concentration is changed to 500 gl-1, the growth rate changes to 0.23h-1. Can you explain the relationship between the sugar concentration and yeast growth rate? Can you predict the growth rate if the concentration is 200 gl-1? Department of Mathematics & Computer Science

5. Linear Assumption Specific growth rate Department of Mathematics & Computer Science

6. How to model the problem • Consider, x the sugar concentration, y the growth rate • Let’s assume that x and y are linearly related. So we can say that y=mx+b. • Use the first condition, x=1000, y=0 0=m*1000+b • Use the second condition, x=500, y=0.23 0.23=m*500+b • So if we list the two equations together, we will have a system of equations. (We will solve it later.) 0.23 = m*500 + b0 = m*1000 + b Department of Mathematics & Computer Science

7. Let’s do math! (Oh no!) • A linear equation in the variables or unknowns x1, x2, ….., xn is an equation that can be written in the form a1x1 + a2 x2 + a3 x3 + ……………+ an xn = b where a1 , a2 , a3,…………… an, b are constants • A collection of linear equations a11x1 + a12 x2 + a13 x3 + ……………+ a1n xn = b1 a21x1 + a22 x2 + a23 x3 + ……………+ a2n xn = b2 : am1x1 + a2 xm2 + a3 xm3 + ……………+ amn xn = bm is called a system oflinear equations Department of Mathematics & Computer Science

8. Solve a system of equations • The Gauss-Jordan elimination method provides a systematic way for solving systems of linear equations. • It involves the repeated use of three basic transformations on a system. The transformations are called elementary operations • Interchange two equations in the system • Multiply an equation by a nonzero number • Replace an equation by itself plus a multiple of another equation • Now let’s solve the yeast problem Department of Mathematics & Computer Science

9. rows columns 2. Matrices and Vectors • A matrix is a set of elements, organized into rows and columns Department of Mathematics & Computer Science

10. Definitions • Let A denote a m x n whose elements are A(i,j): A(1, ); A(2, ), ……A(m, ) denote the first, second, ...., mth row of A, and A( ,1), A( ,2), …..A( ,n) denote the first, second, ..., nth column of A. ----------------------------------------------------------------- • A solution to a system of equations with variables x1, x2, ….., xn consists of values of x1, x2, ….., xn that satisfy each equation in the system Department of Mathematics & Computer Science

11. Basic Operations • Addition, Subtraction, Multiplication Just add elements Just subtract elements Multiply each row by each column Department of Mathematics & Computer Science

12. Multiplication • Is AB = BA? Maybe, but maybe not! • Heads up: multiplication is NOT commutative! Department of Mathematics & Computer Science

13. Transpose • The transpose of a matrix A denoted AT is the matrix obtained by interchanging the rows and columns of A, i.e. (AT)ij = aji e.g. if then Department of Mathematics & Computer Science

14. Symmetric Matrix • A square matrix X is called symmetric if where for all indices i and j AT is the transpose • Example: yes No • This also impliesA-1 ATwhere I is the identity matrix Department of Mathematics & Computer Science

15. A matrix is called skew-symmetric or antisymmetric if its transpose is the same as its negative. XT = - X • Thus the following 3×3 matrix is skew-symmetric: Department of Mathematics & Computer Science

16. Every real symmetric matrix is Hermitian, and therefore all its eigenvalues are real. • Essentially, the property of being symmetric for real matrices corresponds to the property of being Hermitian for complex matrices. • Every square real matrix X can be written uniquely as the sum of a symmetric and a skew-symmetric matrix. This is done in the following way: Department of Mathematics & Computer Science

17. Hermitian Matrix • A Hermitian matrix (or self-adjoint matrix) is a square matrix with complex entries which is equal to its own conjugate transpose — that is, the element in the ith row and jth column is equal to the complex conjugate of the element in the jth row and ith column, for all indices i and j: or written with the conjugate transpose A*: A = A* Example: Department of Mathematics & Computer Science

18. y v x Vector Operations • Vector: 1 x N matrix • Interpretation: a line in N dimensional space • Dot Product, Cross Product, and Magnitude defined on vectors only Department of Mathematics & Computer Science

19. 3. Eigenvalues and Eigenvectors • Two matrices can be multiplied if they are compatible • Example: 1 non-eigenvector and 1 eigenvector • Example of how a scaled eigenvector is still an eigenvector Department of Mathematics & Computer Science

20. What does eigen mean? • In both examples, the amount by which the original vector was scaled after multiplication by the square matrix was the same • In the example, the value was 4. Here, 4 is the eigenvalue associated with that eigenvector. • No matter what multiple of the eigenvector we used before we multiplied it by the square matrix, we would always get 4 times the scaled vector as our result • Eigenvalues are closely related to eigenvectors, • So you see that eigenvectors and eigenvalues always come in pairs. Department of Mathematics & Computer Science

21. Definition • Eigenvectors are a special set of vectors associated with a linear system of equations (i.e., a matrix equation) that are sometimes also known as characteristic vectors, proper vectors, or latent vectors • An eigenvector is a favorite direction for a matrix. • Any square matrix M has at least one nonzero vector v which is mapped in a particularly simple way: M v = λv. • If  λ > 0, then this means that M v is parallel to v. If  λ < 0, they are antiparallel, and if  = 0, then v is in the null space Ker(A). In all cases, M v is just a multiple of v. Department of Mathematics & Computer Science

22. Example • Consider the solution to λis eigenvalue, is eigenvector • In our previous example  (λ – 2) (λ – 1) -6 = 0  λ = 4, λ = 4: (4I –A)x = 0  solution is Department of Mathematics & Computer Science

23. Cont’ • So the vectors associated with e-value λ = 4 are vectors of the form , c is a non-zero scalar • for λ = -1, eigenvector is , s is a nonzero scalar Department of Mathematics & Computer Science

24. Eigenvector • Consider the two column matrices, we have now, So C1is an eigenvector associated with the eigenvalue 4 and C2 is the eigenvector associated with the eigenvalue 1 Department of Mathematics & Computer Science

25. What is the property of Eigenvectors • Next consider the matrix P for which the columns are C1 and C2 i.e. det(P) = -5/2 ≠ 0, so this matrix is invertible. We find next we evaluate P-1AP Department of Mathematics & Computer Science

26. Cont’ Answer: a diagonal matrix • Using matrix multiplication, we obtain • Note it’s usually almost impossible to find An directly from the original form of A for any A Department of Mathematics & Computer Science

27. How do we use Eigenvectors • A measurement of blood and urine from 9 alcoholic people Department of Mathematics & Computer Science

28. Do we need 7 dimensions? Department of Mathematics & Computer Science

29. Do we need 7 dimensions?No • Use Principle Component Analysis (Detailed steps will be in other chapters) • PCA reduces dimensionality and present data via patterns • We can compute (you don’t know how as of now) a matrix that shows the variability of each data (each measurement in the previous table) – covariance matrix Department of Mathematics & Computer Science

30. Principal Component Analysis (PCA) Next we discuss methods of • representation of data using a smaller number of variables, • detection of patterns in noisy gene expression data • identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences -- since patterns in data can be hard to find in data of high dimension, where the luxury of graphical representation is not available, PCA is a powerful tool for analyzing data. • the other main advantage of PCA is that once you have found these patterns in the data, and you compress the data, i.e. by reducing the number of dimensions, without much loss of information. This technique used in image compression, Department of Mathematics & Computer Science

31. Principal Component Analysis • Now we explore the subject of PCA • Recall we said it provided a means of identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences • Suppose we have a population measured on p random variables X1,…,Xp. Note that these random variables represent the p-axes of the Cartesian coordinate system in which the population resides. Department of Mathematics & Computer Science

32. Data Presentation • Better presentation than ordinate axes? • Do we need a p dimension space to view data? • How to find the ‘best’ low dimension space that conveys maximum useful information? • One answer: Find “Principal Components” Department of Mathematics & Computer Science

33. All principal components (PCs) start at the origin of the ordinate axes. First PC is direction of maximum variance from origin Subsequent PCs are orthogonal to 1st PC and describe maximum residual variance Principal Components 30 25 20 Wavelength 2 PC 1 15 10 5 0 0 5 10 15 20 25 30 Wavelength 1 30 25 20 PC 2 Wavelength 2 15 10 5 0 0 5 10 15 20 25 30 Department of Mathematics & Computer Science Wavelength 1

34. The Goal We wish to explain/summarize the underlying variance-covariance structure of a large set of variables through a few linear combinations of these variables. Department of Mathematics & Computer Science

35. Uses: Data Visualization Data Reduction Data Classification Trend Analysis Factor Analysis Noise Reduction ------------------------------ Principal Component Analysis can help in understanding of relationship between sequences of an alignment Method is used for feature extraction Examples: How many unique “sub-sets” are in the sample? How are they similar / different? What are the underlying factors that influence the samples? Which time / temporal trends are (anti)correlated? Which measurements are needed to differentiate? How to best present what is “interesting”? Which “sub-set” does this new sample rightfully belong? Applications Department of Mathematics & Computer Science

36. Statistics We next revisit some basic concepts • Statistics is the science of Data Analysis • The subject is based around the idea that there is some big set of data and we wish to analyze in terms of the relationships between the individual elements of the set • ? What do we do with data – build models of the process that generated the data and in what is known as statistical inference, draw conclusions about this process • Data to be analyzed are derived from genomes • Based on a sequence of symbols from some alphabets (of 20 amino acids) find out what the sequence represents – want to know what protein family a given sequence belongs to Department of Mathematics & Computer Science

37. Standard Deviation • First we review a few measures that we can do on a set of data and what they tell us about the data • Given a set of data X, There are a number of things that we can calculate about a data set. e.g. the mean of the sample X xi is the ith element of the set Standard deviation Department of Mathematics & Computer Science

38. Department of Mathematics & Computer Science

39. Variance • Is another measure of the spread in a data set • Standard deviation and variance only operate on 1 dimension, so that you could only calculate the standard deviation for each dimension of the data set independently of the other dimensions. Department of Mathematics & Computer Science

40. Covariance • Covariance is always measured between 2 dimensions. • the technique is often used to find relationships between dimensions in high-dimensional data sets • If you calculate the covariance between one dimension and itself, you get the variance. • So, if you had a 3-dimensional data set (e.g. x, y, z), then you could measure the covariance between the x and y dimensions, the x and z dimensions, and the y and z dimensions. • Measuring the covariance between x and x , or y and y , or z and z would give the variance of the x , y and z dimensions respectively. Department of Mathematics & Computer Science

41. The formula for covariance is very similar to that for variance. • The formula for variance could be written as: • Thus covariance is Department of Mathematics & Computer Science

42. A useful way to get all the possible covariance values between all the different dimensions is to calculate them all and put them in a matrix. • So, the definition for the covariance matrix for a set of data with dimensions is: • Where Cnxn is a matrix with n rows and n columns, and Dimk is the kth dimension • For a 3-dim data set using dimensions (x,y,z) C is 3x3 Department of Mathematics & Computer Science

43. Department of Mathematics & Computer Science

44. Department of Mathematics & Computer Science

45. Now Back!!How do we use Eigenvectors • A measurement of blood and urine from 9 alcoholic people Department of Mathematics & Computer Science

46. PCA Cont’ • The eigen values/vectors tell us the directionality that most data follows. • Covariance Matrix (showing the variations) Covariance tells whether changes in any two variables move together. Consider two variables x and y. Positive covariance means that high values of yare associated with high values of x. Negative covariance means that high values of yare associated with low values of x. Zero covariance means that there is no association between xand y. Department of Mathematics & Computer Science

47. PCA Cont’ • 7x7 square matrix will have 7 eigenvectors with corresponding eigenvalues • Thus, feature vector = f.v. = (ν1,ν2 ,……, νn) for the data set, • E-values f.v. = (ν1,ν2 ,……, ν7) from • E-vectors: Department of Mathematics & Computer Science

48. Feature 1 vs. Feature 2 Department of Mathematics & Computer Science

49. PCA Procedure • say the original data set was expressed in n-dimension i.e. if n=2, then two axes x, y. • The x, and y axes are perpendicular for the most efficient representation of the data • It is thus important for the eigenvectors to be perpendicular to each other • Obtaining the final data or ‘new’ data as final data = TheChosenRowFeatureVector * original data which now expresses the data in terms of these eigenvectors • In the case of when the new (final) data set has reduced dimensionality, ie. when we have left some of the eigenvectors out, the new data is only in terms of the vectors that we decided to keep. Department of Mathematics & Computer Science

50. PCA Cont’ Observe: • We now have the data in terms of the vectors we have chosen • Our original data set had 7 axes • It is possible to express data in terms of any two axes that you like. • If these axes are perpendicular, then the expression is the most efficient. This was why it was important that eigenvectors are always perpendicular to each other. • We have changed our data from being in terms of the axes x and y, in favor of our 2 eigenvectors. • In the case of when the new data set has reduced dimensionality, ie. when we left some of the eigenvectors out, the new data is only in terms of the vectors that we decided to keep. • To show, plot the final data rotated so e-vectors are axes Department of Mathematics & Computer Science