张量的低秩逼近

1. 张量的低秩逼近 白敏茹 湖南大学数学与计量经济学院 2014-11-15

2. 目 录 • 张量的基本概念 • 张量特征值的计算 • 张量秩1逼近和低秩逼近 • 张量计算软件 • 复张量的最佳秩1逼近和特征值

3. 1. 张量的基本概念 • 张量：多维数组 1阶张量：向量 2阶张量：矩阵 A=(aij) 3阶张量：长方体 A=(aijk)

4. 1. 张量的基本概念 • 张量的秩 秩1张量： 秩1矩阵：A=a bT = (aibj) 张量的秩： 1927年 Hitchcock NP-Hard 可计算 n-rank 其中 表示 张量X的mode-k mode

5. 1. 张量的基本概念 • 张量的低秩逼近：用一个低秩的张量X近似表示张量A 最佳秩R逼近 最佳秩1逼近：R=1 Tucker逼近

6. 1. 张量的基本概念 • 张量的完备化 低秩张量M部分元素 被观察到,其中 是被观察到的元数的指标集. 张量完备化是指:从所观察到的部分元素来恢复逼近低秩张量M

7. 1. 张量的基本概念 • 张量的特征值 H-特征值 2005, Qi Z（E）-特征值 B-特征值 2014，Cui, Dai, Nie US-特征值 2014，Ni, Qi, Bai

8. 2.张量特征值的计算 • 对称非负张量的最大H-特征值的计算： Perron-Frobenius 理论 • Ng, Qi, Zhou 2009， • Chang, Pearson, Zhang 2011， • L. Zhang, L. Qi 2012， • Qi, Q. Yang, Y. Yang 2013 • 对称张量的最大Z-特征值的计算： • The sequential SDPs method [Hu, Huang, Qi 2013] • Sequential subspace projection method[Hao, Cui, Dai. 2014] • Shifted symmetric higher-order power method [Kolda,Mayo 2011] • Jacobian semidefinite relaxations 计算对称张量所有实的B-特征值 • [Cui, Dai, Nie 2014]

9. 2. 张量特征值的计算 • 对称张量的US-特征值的计算： • Geometric measure of entanglement and U-eigenvalues of tensors, SIAM Journal on Matrix Analysis and Applications，[Ni，Qi，Bai 2014] • Complex Shifted Symmetric higher-order power method [Ni, Bai 2014]

10. 3. 张量的秩1逼近和低秩逼近 • 张量的秩1逼近 • 最佳实秩1逼近的计算方法： • 交替方向法(ADM)、截断高阶奇异值分解(T-HOSVD)、 • 高阶幂法(HOPM) 和拟牛顿方法等。----局部解，或稳定点 Nie, Wang[2013] ：半正定松弛方法 ----全局最优解 • 最佳复秩1逼近的计算方法： Ni, Qi，Bai[2014] ：代数方程方法 ----全局最优解

11. 3. 张量的秩1逼近和低秩逼近 • 张量的低秩逼近 • 最佳秩R逼近的计算方法： • 交替最小平方法(ALS) • 最佳Tucker逼近的计算方法： 高阶奇异值（HOSVD），TUCKALS3，t-SVD

12. 4. 张量计算软件 • Matlab, Mathematica, Maple都支持张量计算 • Matlab仅支持简单运算，而对于更一般的运算以及稀疏和结构张量，需要添加软件包（如：N-wayToolbox, CuBatch, PLS Toolbox, Tensor Toolbox）才能支持，其中除PLS Toolbox外，都是免费软件。Tensor Toolbox是支持稀疏张量。 • C++语言软件：HUJI Tensor Library (HTL)，FTensor, Boost Multidimensional Array Library (Boost.MultiArray) • FORTAN语言软件：The Multilinear Engine

13. 5. 复张量的最佳秩1逼近和特征值 [A] Guyan Ni, Liqun Qi and Minru Bai, Geometric measure of entanglement and U-eigenvalues of tensors, SIAM Journal on Matrix Analysis and Applications 2014, 35(1): 73-87 [B] Guyan Ni, Minru Bai, Shifted Power Method for computing symmetric complex tensor US-eigenpairs, 2014, submitted.

14. Basic Definitions 1. A tensor S is called symmetricas its entries s_{i1···id}are invariant under any permutation of their indices. 2. A Z-eigenpair(, u) to a real symmetric tensor S is defined by 2005, Qi uTu 3. An eigenpair(, u) to a real symmetric tensor S is defined by 2011, Kolda and Mayo u*Tu [7] T.G. Kolda and J.R. Mayo, Shifted power method for computing tensor eigenpairs, SIAM Journal on Matrix Analysis and Applications, 32(2011), pp. 1095-1124.

15. 4. The best rank-one tensor approximation problems Assume that T a d-order real tensor. Denote a rank-one tensor . Then the rank-one approximation problem is to minimizes the least-squares cost function The rank-one tensor is said to be the best real rank-one approximation to tensor T. If T is a symmetric real tensor, is said to be the best real symmetric rank-one approximation.

16. Basic results • Friedland [2013] and Zhang et al [2012] showed that the best real rank one approximation to a real symmetric tensor, which in principle can be nonsymmetric, can be chosen symmetric. • ud isthe best real rank-one approximation of T if and only if is a Z-eigenvalue of T with the largest absolute value, (,u) is a Z-eigenpair. [Qi 2011, Friedland2013, Zhang et al 2012] [8] S. Friedland, Best rank one approximation of real symmetric tensors can be chosen symmetric, Frontiers of Mathematics in China, 8(2013), pp. 19-40. [9] X. Zhang, C. Ling and L. Qi, The best rank-1 approximation of a symmetric tensor and related spherical optimization problems, SIAM Journal on Matrix Analysis and Applications 33(2012), pp. 806-821.

17. complex tensors and unitary eigenvalues A d-order complex tensor will be denoted by inner product norm The superscript * denotes the complex conjugate. The superscript Tdenotes transposition. [10] G. Ni, L. Qi and M. Bai, Geometric measure of entanglement and U-eigenvalues of tensors, to appear in SIAM Journal on Matrix Analysis and Applications

18. For A,B ∈ H, define the inner product and norm as inner product norm A rank-one tensor

19. unitary eigenvalue (U-eigenvalue) of T

20. Denote by Sym(d, n) all symmetric d-order n-dimensional tensors Let x ∈Cn. Simply denote the rank-one tensor Define We call a number ∈C a unitary symmetric eigenvalue (US-eigenvalue)of S if and a nonzero vector

21. The largest |λ| is the entanglement eigenvalue. The corresponding rank-one tensor ⊗di =1x is the closest symmetric separable state. Theorem 1. Assume that complex d-order tensors Then b) all U-eigenvalues are real numbers; c) the US-eigenpair (, x) to a symmetric d-order complex tensor S can also be defined by the following equation system (1) or

22. 3.1. US-eigenpairs of symmetric tensors Case d=2: Theorem 3. (Takagi’s factorization) Let A ∈Cn×nbe a complex symmetric tensor. Then there exists a unitary matrix U ∈Cn×nsuch that Theorem 4. Let A ∈Cn×nbe a complex symmetric tensor. Let U ∈Cn×nbe a unitary matrix such that Let ei= (0, · · · , 0, 1, 0, · · · , 0)T , i = 1, · · · , n. Then both are US-eigenpairs of A. and The number of distinct US-eigenvalues is at most 2n.

23. Theorem 5. If 1 = · · · = k > k+1, 1 ≤ k ≤ n, then the set of all US-eigenvectors with respect to 1 is the set of all US-eigenvectors with respect to −λ1 is

24. 3.2. US-eigenpairs of symmetric tensors Case d  3 • The problem of finding eigenpairs is equivalent to solving a polynomial system [8] S. Friedland, Best rank one approximation of real symmetric tensors can be chosen symmetric, Frontiers of Mathematics in China, 8(2013), pp. 19-40.

25. 3.2. US-eigenpairs of symmetric tensors Case d  3 Theorem 2. Assume that a complex d-order n-dimension symmetric tensor S ∈ Sym(d, n). Then a) if d ≥3, d is an odd integer, and  0, then the system (1) is equivalent to (2) and the number of US-eigenpairs of (1) is the double of the number of solutions of (2); b) if d ≥3, d is an even integer, and  0, then the system (1) is equivalent to (3) and the number of US-eigenpairs of (1) is equal to the number of solutions of (3).

26. 3.2. US-eigenpairs of symmetric tensors Case d  3 Theorem 6. Let d ≥ 3, n ≥2 be integers, S ∈ Sym(d, n). If (2) has finitely many solutions, then a) if d is odd , the number of non-zero solutions of (2) is at most b) if d is even, the number of non-zero solutions of (3) is at most c) S has at most distinct nonzero US-eigenvalues; d) for nonzero US-eigenvalues, all the US-eigenpairs of S are as follows where x is a solution of (2).

27. Note. 1. Let S be the symmetric 2 ×2 ×2 ×2 tensor whose non-zero entries are S1111 = 2, S1112 = −1, S1122 = −1, S1222 = −2, S2222 = 1. The number of non-zero solutions of the equation system (2) is 40 which shows that the bound is tight. Note. 2. Cartwright and Sturmfels (2013) showed that every symmetric tensor has finite E-eigenvalues. At the same time, they indicated that the magnitudes of the eigenvalues with ||x|| = 1 may still be an infinite set (See Example 5.8 of [Cartwright and Sturmfels (2013) ]), which implies that the system S x d−1 = x has infinite non-zero solutions, where S is a symmetric 3 ×3 ×3 tensor whose non-zero entries are S111 = 2, S122 = S212 = S221 = S133 = S313 = S331 = 1. [11] D. Cartwright and B. Sturmfels, The number of eigenvalues of a tensor, Linear Algebra and its Applications, 438(2013), pp. 942-952

28. Note. 3. Let S be the symmetric 3×3×3 tensor as in Note 2. Then x = for all 0 < a < 1 are non-zero solutions of Sx d−1 = x*. It implies that (2) may have infinite non-zero solutions.

29. 4. Best symmetric rank-one approximation of symmetric tensors Theorem 7. Let S be a symmetric complex tensor. Let be a US-eigenvalue of S. Then a) − is also a US-eigenvalue of S ; b) G(S) = max. Case d=2 Theorem 8. Let A ∈Cn×nbe a complex symmetric matrix. Then for all x ∈ UEV (A, 1) ∪ UEV (A,− 1) and ∈C with || = 1, (x) ⊗(x) are best symmetric rank-one approximation of A.

30. Case d ≥3 The best symmetric rank-one approximation problem is to find a unit-norm vector x ∈Cn, such that By Theorem 7, introducing the US-eigenvalue method, Q1 is equivalent to the following problem

31. Theorem 9. Let S ∈ Sym(d, n). Then a) the best symmetric rank-one approximation problem is equivalent to the following optimization problem rank-one approximation of S for each • The problem of finding eigenpairs is equivalent to solving a polynomial system

32. Let x = y + z−1, y, z ∈ Rn. Then Q3 is equivalent to the following problem Example 1. Assume that S is a real symmetric tensor with d = 3 and n = 2. Then Q4 is equivalent to the following optimization problem

33. Table1. US-eigenpairs of S with S111 = 2, S112 = 1, S122 = −1, S222 = 1. The best real rank-one approximation is also the best complex rank-one approximation.

34. The absolute-value largest of Z-eigenvalues is not its largest US-eigenvalue.

35. By observing numerical examples, we find the following results: • The best real rank-one approximation is sometimes also the best complex rank-one approximation even if the tensor is not a symmetric nonnegative real tensor, see Table 1. • The absolute-value largest of Z-eigenvalues is sometimes not its largest US-eigenvalue, see Table 2. Question 1: What is the necessary and sufficient condition for the equality of the largest absolute Z-eigenvalue and the largest US-eigenvalue to a real symmetric tensor?

36. 谢谢大家！