矩阵大全

【实例截图】

【核心代码】

Contents
1 Basics 6
1.1 Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 The Special Case 2x2 . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Derivatives 8
2.1 Derivatives of a Determinant . . . . . . . . . . . . . . . . . . . . 8
2.2 Derivatives of an Inverse . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Derivatives of Eigenvalues . . . . . . . . . . . . . . . . . . . . . . 10
2.4 Derivatives of Matrices, Vectors and Scalar Forms . . . . . . . . 10
2.5 Derivatives of Traces . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.6 Derivatives of vector norms . . . . . . . . . . . . . . . . . . . . . 14
2.7 Derivatives of matrix norms . . . . . . . . . . . . . . . . . . . . . 14
2.8 Derivatives of Structured Matrices . . . . . . . . . . . . . . . . . 14
3 Inverses 17
3.1 Basic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 Exact Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3 Implication on Inverses . . . . . . . . . . . . . . . . . . . . . . . . 20
3.4 Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.5 Generalized Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.6 Pseudo Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4 Complex Matrices 24
4.1 Complex Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2 Higher order and non-linear derivatives . . . . . . . . . . . . . . . 26
4.3 Inverse of complex sum . . . . . . . . . . . . . . . . . . . . . . . 27
5 Solutions and Decompositions 28
5.1 Solutions to linear equations . . . . . . . . . . . . . . . . . . . . . 28
5.2 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . 30
5.3 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . 31
5.4 Triangular Decomposition . . . . . . . . . . . . . . . . . . . . . . 32
5.5 LU decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.6 LDM decomposition . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.7 LDL decompositions . . . . . . . . . . . . . . . . . . . . . . . . . 33
6 Statistics and Probability 34
6.1 Definition of Moments . . . . . . . . . . . . . . . . . . . . . . . . 34
6.2 Expectation of Linear Combinations . . . . . . . . . . . . . . . . 35
6.3 Weighted Scalar Variable . . . . . . . . . . . . . . . . . . . . . . 36
7 Multivariate Distributions 37
7.1 Cauchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.2 Dirichlet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.3 Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.4 Normal-Inverse Gamma . . . . . . . . . . . . . . . . . . . . . . . 37
7.5 Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.6 Multinomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 3
CONTENTS CONTENTS
7.7 Student’s t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.8 Wishart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
7.9 Wishart, Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
8 Gaussians 40
8.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
8.2 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
8.3 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
8.4 Mixture of Gaussians . . . . . . . . . . . . . . . . . . . . . . . . . 44
9 Special Matrices 46
9.1 Block matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
9.2 Discrete Fourier Transform Matrix, The . . . . . . . . . . . . . . 47
9.3 Hermitian Matrices and skew-Hermitian . . . . . . . . . . . . . . 48
9.4 Idempotent Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 49
9.5 Orthogonal matrices . . . . . . . . . . . . . . . . . . . . . . . . . 49
9.6 Positive Definite and Semi-definite Matrices . . . . . . . . . . . . 50
9.7 Singleentry Matrix, The . . . . . . . . . . . . . . . . . . . . . . . 52
9.8 Symmetric, Skew-symmetric/Antisymmetric . . . . . . . . . . . . 54
9.9 Toeplitz Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
9.10 Transition matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 55
9.11 Units, Permutation and Shift . . . . . . . . . . . . . . . . . . . . 56
9.12 Vandermonde Matrices . . . . . . . . . . . . . . . . . . . . . . . . 57
10 Functions and Operators 58
10.1 Functions and Series . . . . . . . . . . . . . . . . . . . . . . . . . 58
10.2 Kronecker and Vec Operator . . . . . . . . . . . . . . . . . . . . 59
10.3 Vector Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
10.4 Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
10.5 Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
10.6 Integral Involving Dirac Delta Functions . . . . . . . . . . . . . . 62
10.7 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
A One-dimensional Results 64
A.1 Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
A.2 One Dimensional Mixture of Gaussians . . . . . . . . . . . . . . . 65
B Proofs and Details 66
B.1 Misc Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 4
CONTENTS CONTENTS
Notation and Nomenclature
A Matrix
A ij Matrix indexed for some purpose
A i Matrix indexed for some purpose
A ij Matrix indexed for some purpose
A n Matrix indexed for some purpose or
The n.th power of a square matrix
A −1 The inverse matrix of the matrix A
A The pseudo inverse matrix of the matrix A (see Sec. 3.6)
A 1/2 The square root of a matrix (if unique), not elementwise
(A) ij The (i,j).th entry of the matrix A
A ij The (i,j).th entry of the matrix A
[A] ij The ij-submatrix, i.e. A with i.th row and j.th column deleted
a Vector (column-vector)
a i Vector indexed for some purpose
a i The i.th element of the vector a
a Scalar
?z Real part of a scalar
?z Real part of a vector
?Z Real part of a matrix
?z Imaginary part of a scalar
?z Imaginary part of a vector
?Z Imaginary part of a matrix
det(A) Determinant of A
Tr(A) Trace of the matrix A
diag(A) Diagonal matrix of the matrix A, i.e. (diag(A)) ij = δ ij A ij
eig(A) Eigenvalues of the matrix A
vec(A) The vector-version of the matrix A (see Sec. 10.2.2)
sup Supremum of a set
||A|| Matrix norm (subscript if any denotes what norm)
A T Transposed matrix
A −T The inverse of the transposed and vice versa, A −T = (A −1 ) T = (A T ) −1 .
A ∗ Complex conjugated matrix
A H Transposed and complex conjugated matrix (Hermitian)
A ◦ B Hadamard (elementwise) product
A ⊗ B Kronecker product
0 The null matrix. Zero in all entries.
I The identity matrix
J ij The single-entry matrix, 1 at (i,j) and zero elsewhere
Σ A positive definite matrix
Λ A diagonal matrix