Introduction to tensor, tensor factorization and its applications

1. Introduction to tensor, tensor factorization and its applications Mu Li iPAL Group Meeting Sept. 17, 2010

2. Outline • Basic concepts about tensor 1. What’s tensor? Why tensor and tensor factorization? 2. Tensor multiplication 3. Tensor rank • Tensor factorization 1. CANDECOMP/PARAFAC factorization 2. Tucker factorization • Applications of tensor factorization • Conclusion

3. What’s tensor? Why tensor and tensor factorization? • Definition: a tensor is a multidimensional array which is an extension of matrix. • Tensor can happen in daily life. • In order to facilitate information mining from tensor and tensor processing, storage, tensor factorization is often needed. • Three-way tensor:

4. A tensor is a multidimensional array

5. Fiber and slice

6. Tensor unfoldings: Matricization and vectorization • Matricization: convert a tensor to a matrix • Vectorization: convert a tensor to a vector

7. Tensor multiplication: the n-mode product: multiplied by a matrix • Definition:

8. Tensor multiplication: the n-mode product: multiplied by a vector • Definition: • Note: multiplying by a vector reduces the dimension by one.

9. Rank-one Tensor and Tensor rank • Rank-one tensor: • Example: • Tensor rank: smallest number of rank-one tensors that can generate it by summing up. • Differences with matrix rank: 1. tensor rank can be different over R and C. 2. Deciding tensor rank is an NP problem that no straightforward algorithm can solve it.

10. Tensor factorization: CANDECOMP/PARAFAC factorization(CP) • Tensor factorization: an extension of SVD and PCA of matrix. • CP factorization: • Uniqueness: CP of tensor(higher-order) is unique under some general conditions. • How to compute: Alternative Least Squares(ALS), fixing all but one factor matrix to which LS is applied.

11. Differences between matrix SVD and tensor CP • Lower-rank approximation is different between matrix and higher-order tensor • Matrix: • Not true for higher-order tensor

12. Tensor factorization: Tucker factorization • Tucker factorization: • For three-way tensor, Tucker factorization has three types: • Tucker3: • Tucker2: • Tucker1:

13. Three types of Tucker factorization

14. Tucker factorization • Uniqueness: Unlike CP, Tucker factorization is not unique. • How to compute: Higher-order SVD(HOSVD), for each n, Rn:

15. Applications of Tensor factorization • A simple application of CP:

16. Apply CP to reconstruct a MATLAB logo from noisy data

17. Apply Tucker3 to do data reconstruction from noise

18. Apply Tucker3 to do cluster analysis

19. Conclusion • Tensor is a multidimensional array which is an extension of matrix that arises frequently in our daily life such as video, microarray data, EEG data, etc. • Tensor factorization can be considered higher-order generalization of matrix SVD or PCA, but they also have much differences, such as NP essential of deciding higher-order tensor rank, non-optimal property of higher-order tensor factorization. • There are still many other tensor factorizations, such as block-oriented decomposition, DEDICOM, CANDELINC. • Tensor factorizations have wide applications in data reconstruction, cluster analysis, compression etc.

20. References • Kolda, Bader, Tensor decompositions and applications. • Martin, an overview of multilinear algebra and tensor decompositions. • Cichocki, etc., nonnegative matrix and tensor factorizations.