1 / 54

Bilinear Least Squares and Parallel Factor Analysis

Bilinear Least Squares and Parallel Factor Analysis. By: S.M. Sajjadi Islamic Azad University, Parsian Branch, Parsian,Iran. Contents. Constructing Three-way Array. Spatial Parts in the Three-way Array. Matricizing Three-way Array. Elementary Operations.

wyatt-castro
Download Presentation

Bilinear Least Squares and Parallel Factor Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bilinear Least Squares and Parallel Factor Analysis By: S.M. Sajjadi Islamic Azad University, Parsian Branch, Parsian,Iran

  2. Contents Constructing Three-way Array Spatial Parts in the Three-way Array Matricizing Three-way Array Elementary Operations Introducing of Three-mode Factor Analysis (3MFA) , Bilinear Least Squares PARAFAC algorithm 1

  3. Our Goal Writing the Program of PARAFAC 2

  4. The Rules for Pictorial Representation Scalar Vector Three-way array Matrix a = A (I×J) = a (I×1) = A (I×J×K) = 3

  5. 280 290 300 320 21.6 8.64 2.7 14.4 5.76 1.8 340 50.4 20.16 6.3 33.6 13.44 4.2 360 36 14.4 4.5 24 9.6 3 380 28.8 11.52 3.6 EEM EEM 19.2 7.68 2.4 c1 c2 Data Matrix for Two Mixture of Analytes A and B 280 290 300 320 340 360 380 4

  6. 14.4 5.76 1.8 21.6 8.64 2.7 33.6 13.44 4.2 50.4 20.16 6.3 24 9.6 3 36 14.4 4.5 19.2 7.68 2.4 28.8 11.52 3.6 Constructing Three-way Data Array by Stacking Two-way Data X( : , : , 1 ) = X1 X(4×3×2) X( : , : , 2 ) = X2 X(2×4×3)?? X(4×2×3)?? For two-way arrays it is useful to distinguish between special parts of the array, such as rows and columns. What are spatial parts in the three-way array? 5

  7. Rows, Columns and Tubes 2 3 Row Column 4 Tube 6

  8. Rows, Columns and Tubes 2 3 Column X( : , j , k ) 4 xjk(4×1) 6

  9. Rows, Columns and Tubes 2 3 Row Column X( i , : , k ) X( : , j , k ) 4 xik(3×1) xjk(4×1) 6

  10. Rows, Columns and Tubes 2 3 Row Column X( i , : , k ) X( : , j , k ) 4 xik(3×1) xjk(4×1) X( i , j , : ) Tube xij(2×1) 6

  11. Horizontal, Vertical and Frontal slices 2 3 Horizental Vertical 4 Frontal 7

  12. Horizontal, Vertical and Frontal slices 2 3 Horizental X( i , : , : ) 4 7

  13. Horizontal, Vertical and Frontal slices 2 3 Horizental Vertical X( : , j , : ) X( i , : , : ) 4 7

  14. Horizontal, Vertical and Frontal slices 2 3 Horizental Vertical X( : , j , : ) X( i , : , : ) 4 Frontal X( : , : , k ) 7

  15. Practice 1 There are five EEMs of different samples that contain two analytes. Please construct three kinds of three-way data array, i.e., consider each EEM as frontal, horizontal and vertical slice. 8

  16. Matricizing Three-way in First Direction 2 3 4 9

  17. Matricizing Three-way in First Direction 2 3 3 4 4 9

  18. Matricizing Three-way in First Direction 2 3 3 6 X( : , : ) 4 4 9

  19. Matricizing Three-way in First Direction 6 2 3 3 6 ?? X( : , : ) 4 4 ? 4 3 2 6 2 X ( : , : ) permute ( X , [1 3 2] ) 4 4 9

  20. Matricizing Three-way in Second Direction 2 3 3 2 8 4 permute ( X , [2 …) 1- Which dimension should be matricized? 2- Which dimension should be arranged one by one ? permute ( X , [2 3 1] ) Matrisizing : X ( : , : ) 10

  21. Practice 2 • There are five EEMs of different samples that contain two analytes. • Please construct three kinds of three-way data array, i.e., consider each EEM as frontal, horizontal and vertical slices. Please matricize one of the constructed three-way data in third direction (two kinds). 11

  22. Elementary Operations Vector multiplication • Inner product = scalar aTb = scalar: = I I J J • Outer product = Martix = I I 12

  23. Elementary Operations Matrix multiplication J K AB = matrix K J = I I Vec-operator Vec of matrix A is the IJ vector . . . vectorized . . . . IJ 13

  24. Elementary Operations In order to concisely describe multi-way models, the usual matrix product is not sufficient. Three other types of matrix products are introduced: Tucker Kronecker product PARAFAC Khatri–Rao product  Hadamard product Weighted PARAFAC * 14

  25. 3× 4× 3B 4B 7× 3× 7B 3B A B = 5B 8B 4B 12B 5× 8× 4× 12× Kronecker product: Simple Example = kron(A,B) 15

  26. 3B 4B 7B 3B A B = = 5B 8B 4B 12B Kronecker product: Simple Example 16

  27. Kronecker product B(K×M) , A(I×J) 17

  28. A B = 3 4 4 4 4 4 7 1.6 1.6 1.6 1.6 1.6 4 5 1.2 1.2 1.2 1.2 1.2 0.5 0.5 0.5 0.5 0.5 3 4 3.6 3.6 3.6 3.6 3.6 8 4.9 4.9 4.9 4.9 4.9 12 Khatri-Rao Product: Simple Example of 3× 4× A= 3× 7× =  B = 8× 5× kron(A(:,1),B(:,1)) 12× 4× 18 kron(A(:,2),B(:,2))

  29. . Khatri-Rao Product A and B are partitioned matrices with an equal number of partitions. A =[a1, a2 ,…, an] B =[b1, b2 ,…, bn]; A B = 19

  30. Hadamard Product Hadamard or element wise product, which is defined formatrices A and B of equal size ( I × J ) 20

  31. Addition and Subtraction of Multi-way Array K K K J J J + = I I I 21

  32. Addition and Subtraction of Multi-way Array K K K J J J - = I I I 21

  33. Analysis of Three-way Data by Three-mode Factor Analysis (3MFA) K C R J Q B G K P J A + E I I If P=Q=Rand all of the elements of G is 0 except superdiagonal element that equal one, data is trilinear and method is called PARAFAC. 22

  34. Constructing Slices of Trilinear Data C K N N J B N = + E A I 2 If N=2: B 2 Xk = A ck1 ck2 23

  35. Constructing Slices of Trilinear Data 2 B 2 Frontal Slices Xk = A Dk Xk = ADkB = ck1a1b1 + ck2a2b2 Across all slices Xk , the components ar and br remain the same, only their weights dk1 , . . . , dk2 are different. Vertical Slices Horizental Slices 24

  36. Practice 3 There are excitation, emission and concentration matrix of two analytes. Please do Khatri-Rao product of excitation and emission matrix. Please simulate a trilinear data by these matrices. 25

  37. Ordinary Least Squares Calibration step: S X = C Sensitivity Matrix S = C+X Prediction Step: = c = S+ x 26

  38. Three-way Data and ?? Least Squares Frontal Slices Xk = ADkB = ck1a1b1 + ·· ·+ckRaRbR We need to estimate the parameters A and B of the calibration model, which we can then use for future predictions. One possibility is bilinear least squares (BLLS) 27

  39. Algorithm of BLLS Sample1: [c11 c12] (4×3) Z(1) [c21 c22] Sample2: Z(2) (4×3) [c31 c32] Sample3: Z(3) (4×3) 1.Vectorizing of Matrices . . . . 28

  40. Algorithm of BLLS 2. Folding of Vectorized Matrices Folding 3. Obtaining Sensitivity Matrix = Sensitivity Matrix S = C+X • For unknown matrix Z0calculate 29

  41. What Is Sensitivity Matrix in BLLS? Only contribution of first component Only contribution of another of component Matricized C K 2 2 J B 2 = A SVD SVD a1,b1 a2,b2 I 30

  42. A B = . What Is Sensitivity Matrix in BLLS? Kronocker product J I a1 b1 Kronocker product J I a2 b2 Sensitivity matrix is a transpose of Khatri-Rao product of A and B 31

  43. PARAFACAlgorithm Alternating least squares PARAFAC algorithm Algorithms for fitting the PARAFAC model are usually based on alternating least squares. This is advantageous because the algorithm is simple to implement, simple to incorporate constraints in, and because it guarantees convergence. However, it is also sometimes slow. 32

  44. PARAFACAlgorithm The solution to the PARAFAC model can be foundby alternating least squares (ALS) by successively assuming the loadings in two modes known and then estimating the unknown set of parameters of the last mode. • Determining the rank of three-way array • The PARAFAC algorithm begins with an initial guess of the two loading modes 33

  45. PARAFACAlgorithm • Suppose initial estimates of B and C loading modes are given Step1.Determining of A profile K JK Matricizing J I I N = N JK I 34

  46. JK N = N JK I I A = XZ+ PARAFAC - Step1.Determining of A profile N N N C Khatri-Rao K B CB=ZA J JK 35

  47. PARAFAC – Step2.Determining of B profile Matricizing X (I×J×K) X (J×IK) IK N N = IK J J X(J×IK)= B(J×N)(CA)T= B ZBT B =XZB+ 36

  48. PARAFAC – Step3.Determining of C profile Matricizing X (I×J×K) X (K×IJ) IK N = N IJ K K X(K×IJ)= C(K×N)(BA)T= C ZCT C =XZC+ 37

  49. PARAFAC – Step4-5.Calculating Relative fit 4-1. Reconstructing Three-way Array from obtained A and B and C profiles 4-2.Calculating the norm of residual array 5. Go to step 1 until relative change in fit is small. 38

More Related