Summary of all matrix operations

http://www.statmethods.net/advstats/matrix.html

Create a matrix

• an example from ?matrix

mdat <- matrix(c(1,2,3, 11,12,13), nrow = 2, ncol = 3, byrow = FALSE,
               dimnames = list(c("row1", "row2"),
                               c("C.1", "C.2", "C.3")))
mdat

##      C.1 C.2 C.3
## row1   1   3  12
## row2   2  11  13

• an alternative way

mdat <- c(1,2,3, 11,12,13)
dim(mdat) <- c(2,3)
dimnames(mdat) <- list(c("row1", "row2"), c("C.1", "C.2", "C.3"))
mdat

##      C.1 C.2 C.3
## row1   1   3  12
## row2   2  11  13

Appending a column or a row to a matrix

dim(mdat)

## [1] 2 3

• appending a column

mat2 <- cbind(mdat, c(3,4))
dim(mat2)

## [1] 2 4

• appending a row

mat3 <- rbind(mdat, c(2,5,8))
dim(mat3)

## [1] 3 3

Subset of a matrix

• by numeric index

mdat[1,1]

## [1] 1

mdat[1,1:2]

## C.1 C.2 
##   1   3

mdat[1,]

## C.1 C.2 C.3 
##   1   3  12

mdat[1,-1]

## C.2 C.3 
##   3  12

mdat[,]

##      C.1 C.2 C.3
## row1   1   3  12
## row2   2  11  13

• by names

dimnames(mdat); rownames(mdat); colnames(mdat)

## [[1]]
## [1] "row1" "row2"
## 
## [[2]]
## [1] "C.1" "C.2" "C.3"

## [1] "row1" "row2"

## [1] "C.1" "C.2" "C.3"

mdat["row1", "C.1"]

## [1] 1

mdat["row2", c("C.2", "C.3")]

## C.2 C.3 
##  11  13

mdat["row1",-2] ## mixed usage

## C.1 C.3 
##   1  12

Transponse of a Matrix

• t()

mat1 <- matrix(1:6,nrow=2,ncol=3)
mat1

##      [,1] [,2] [,3]
## [1,]    1    3    5
## [2,]    2    4    6

t(mat1)

##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4
## [3,]    5    6

Element-wise Multiplication by a Scalar/Vector/Matrix

• by a scalar

mdat * 3

##      C.1 C.2 C.3
## row1   3   9  36
## row2   6  33  39

• by a column vector

mdat * c(1,2)

##      C.1 C.2 C.3
## row1   1   3  12
## row2   4  22  26

• by a matrix of the same dimension

mdat * matrix(1:6,nrow=2,ncol=3)

##      C.1 C.2 C.3
## row1   1   9  60
## row2   4  44  78

Addition/Subtraction by a Scalar/Vector/Matrix

• by a scalar

mdat + 3

##      C.1 C.2 C.3
## row1   4   6  15
## row2   5  14  16

• by a column vector

mdat - c(1,2)

##      C.1 C.2 C.3
## row1   0   2  11
## row2   0   9  11

• by a matrix of the same dimension

mdat + matrix(1:6,nrow=2,ncol=3)

##      C.1 C.2 C.3
## row1   2   6  17
## row2   4  15  19

Matrix Multiplication

• %*%

mat1 <- matrix(1:6,nrow=2,ncol=3)
mat1

##      [,1] [,2] [,3]
## [1,]    1    3    5
## [2,]    2    4    6

mat2 <- matrix(1:6,nrow=3,ncol=2)
mat2

##      [,1] [,2]
## [1,]    1    4
## [2,]    2    5
## [3,]    3    6

mat1 %*% mat2

##      [,1] [,2]
## [1,]   22   49
## [2,]   28   64

mat2 %*% mat1

##      [,1] [,2] [,3]
## [1,]    9   19   29
## [2,]   12   26   40
## [3,]   15   33   51

crossProd

• crossprod(A,B): $A^\top B$
• crossprod(A) $A^\top A$

A <- matrix(c(1,2), ncol =1)
B <- matrix(c(2,2), ncol=1)

A 

##      [,1]
## [1,]    1
## [2,]    2

B

##      [,1]
## [1,]    2
## [2,]    2

crossprod(A,B)

##      [,1]
## [1,]    6

crossprod(t(A),t(B))

##      [,1] [,2]
## [1,]    2    2
## [2,]    4    4

crossprod(A)

##      [,1]
## [1,]    5

crossprod(t(A))

##      [,1] [,2]
## [1,]    1    2
## [2,]    2    4

Diagonal Matrix

• diag()
  • if input is a square matrix, return the diagnal element.
  • if input is a vector, return a diagnal matrix.
  • if input is a scalar (i.e. k), return this creates a k x k identity matrix

S <- matrix(1:9,3,3)
S

##      [,1] [,2] [,3]
## [1,]    1    4    7
## [2,]    2    5    8
## [3,]    3    6    9

diag(S)

## [1] 1 5 9

diag(c(1,4,7))

##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    4    0
## [3,]    0    0    7

diag(3)

##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

Inverse of a Matrix

S <- matrix(c(4,4,-2,2,6,2,2,8,4),3,3)
SI <- solve(S)
SI

##      [,1] [,2] [,3]
## [1,]  1.0 -0.5  0.5
## [2,] -4.0  2.5 -3.0
## [3,]  2.5 -1.5  2.0

S %*% SI

##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

SI %*% S

##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

More about solve

• solve(A, b) Returns vector x in the equation $b = Ax$ (i.e., $A^{-1}b$)

set.seed(32608)
(A <- matrix(rnorm(9),3,3))

##             [,1]       [,2]       [,3]
## [1,]  0.07699352  0.3381365 -1.9363229
## [2,] -0.56610906 -0.3786229 -0.5720703
## [3,]  0.80627284  0.7446219  0.7198154

(b <- runif(3))

## [1] 0.5756702 0.8367205 0.2964995

(x <- solve(A,b))

## [1] -7.2976775  7.5871849  0.7374597

A %*% x

##           [,1]
## [1,] 0.5756702
## [2,] 0.8367205
## [3,] 0.2964995

Moore-Penrose Generalized Inverse

• Generalized inverse of a matrix $A\in \mathbb{R}^{n\times m}$ is defined as $A^g \in \mathbb{R}^{m\times n}$: $$AA^gA = A$$
• Regular inverse is a special case of Generalized Inverse when $m = n$ and $A$ is non-singular.

library(MASS)
set.seed(32608)
A <- matrix(rnorm(9),3,3)
ginv(A)

##            [,1]      [,2]       [,3]
## [1,]  0.7013869 -7.703428 -4.2355174
## [2,] -0.2457031  7.389853  5.2121071
## [3,] -0.5314604  0.984167  0.7417649

(A2 <- A[,1:2])

##             [,1]       [,2]
## [1,]  0.07699352  0.3381365
## [2,] -0.56610906 -0.3786229
## [3,]  0.80627284  0.7446219

ginv(A2)

##           [,1]      [,2]      [,3]
## [1,] -2.572491 -1.640808 0.3338684
## [2,]  3.079399  1.232375 0.5712269

A2 %*% ginv(A2) %*% A2

##             [,1]       [,2]
## [1,]  0.07699352  0.3381365
## [2,] -0.56610906 -0.3786229
## [3,]  0.80627284  0.7446219

Eigen value decomposition

• reference: https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix

• A (non-zero) vector $v$ of dimension $N$ is an eigenvector of a square $A\in \mathbb{R}^{N\times N}$ if it satisfies the linear equation $$Av = \lambda v$$

• Let A be a square (N×N) matrix with N linearly independent eigenvectors, $q_i (i = 1, \ldots, N)$. Then $A$ can be factorized as $$A = Q \Lambda Q^{-1}$$

A0 <- c(3,1,1,
       1,3,2,
       1,2,3)
A <- matrix(A0,3,3)
eigen(A)

## $values
## [1] 5.732051 2.267949 1.000000
## 
## $vectors
##            [,1]       [,2]       [,3]
## [1,] -0.4597008  0.8880738  0.0000000
## [2,] -0.6279630 -0.3250576 -0.7071068
## [3,] -0.6279630 -0.3250576  0.7071068

Q <- eigen(A)$vectors
lambda <- eigen(A)$values

Q %*% t(Q)

##               [,1]          [,2]          [,3]
## [1,]  1.000000e+00 -5.551115e-17  0.000000e+00
## [2,] -5.551115e-17  1.000000e+00 -2.220446e-16
## [3,]  0.000000e+00 -2.220446e-16  1.000000e+00

Q %*% diag(lambda) %*% ginv(Q)

##      [,1] [,2] [,3]
## [1,]    3    1    1
## [2,]    1    3    2
## [3,]    1    2    3

Singular value decomposition (SVD)

• https://en.wikipedia.org/wiki/Singular_value_decomposition

• Singular value decomposition of $M \in \mathbb{R}^{m \times n}$ can be factorized as $U\Sigma V^\top$
  • $U \in \mathbb{R}^{m \times m}$ such that $UU^\top = I_m$.
  • $\Sigma \in \mathbb{R}^{m \times n}$ rectangular diagonal matrix.
  • $V \in \mathbb{R}^{n \times n}$ such that $VV^\top = I_n$.
• Equivalently, if $m \le n$, $M \in \mathbb{R}^{m \times n}$, $M = U\Sigma V^\top$
  • $U \in \mathbb{R}^{m \times m}$ such that $UU^\top = I_m$.
  • $\Sigma \in \mathbb{R}^{m \times m}$ rectangular diagonal matrix.
  • $V \in \mathbb{R}^{n \times m}$ such that $V^\top V= I_m$.

M <- matrix(1:6,2,3) 
M

##      [,1] [,2] [,3]
## [1,]    1    3    5
## [2,]    2    4    6

svd(M)

## $d
## [1] 9.5255181 0.5143006
## 
## $u
##            [,1]       [,2]
## [1,] -0.6196295 -0.7848945
## [2,] -0.7848945  0.6196295
## 
## $v
##            [,1]       [,2]
## [1,] -0.2298477  0.8834610
## [2,] -0.5247448  0.2407825
## [3,] -0.8196419 -0.4018960

svdRes <- svd(M)
svdRes$u %*% diag(svdRes$d) %*% t(svdRes$v)

##      [,1] [,2] [,3]
## [1,]    1    3    5
## [2,]    2    4    6

Determinant of a Matrix

• det()

S <- matrix(c(4,4,-2,2,6,2,2,8,4),3,3)
det(S)

## [1] 8

• rank (rank function has been implemented for the purpose of rank of a vector)
• To get the rank of a matrix:
  • Perform QR decomposition first.

S <- matrix(c(4,4,-2,2,6,2,2,8,4),3,3)
QR <- qr(S)
QR$rank

## [1] 3

Column and Row Means

• row mean

S <- matrix(c(4,4,-2,2,6,2,2,8,4),3,3)
apply(S,1,mean)

## [1] 2.666667 6.000000 1.333333

rowMeans(S)

## [1] 2.666667 6.000000 1.333333

• column mean

S <- matrix(c(4,4,-2,2,6,2,2,8,4),3,3)
apply(S,2,mean)

## [1] 2.000000 3.333333 4.666667

colMeans(S)

## [1] 2.000000 3.333333 4.666667

Generate code

knitr::purl("matrix.rmd", output = "matrix.R ", documentation = 2)

## 
## 
## processing file: matrix.rmd

## 
  |                                                                       
  |                                                                 |   0%
  |                                                                       
  |.                                                                |   2%
  |                                                                       
  |..                                                               |   4%
  |                                                                       
  |....                                                             |   6%
  |                                                                       
  |.....                                                            |   7%
  |                                                                       
  |......                                                           |   9%
  |                                                                       
  |.......                                                          |  11%
  |                                                                       
  |........                                                         |  13%
  |                                                                       
  |..........                                                       |  15%
  |                                                                       
  |...........                                                      |  17%
  |                                                                       
  |............                                                     |  19%
  |                                                                       
  |.............                                                    |  20%
  |                                                                       
  |..............                                                   |  22%
  |                                                                       
  |................                                                 |  24%
  |                                                                       
  |.................                                                |  26%
  |                                                                       
  |..................                                               |  28%
  |                                                                       
  |...................                                              |  30%
  |                                                                       
  |....................                                             |  31%
  |                                                                       
  |......................                                           |  33%
  |                                                                       
  |.......................                                          |  35%
  |                                                                       
  |........................                                         |  37%
  |                                                                       
  |.........................                                        |  39%
  |                                                                       
  |..........................                                       |  41%
  |                                                                       
  |............................                                     |  43%
  |                                                                       
  |.............................                                    |  44%
  |                                                                       
  |..............................                                   |  46%
  |                                                                       
  |...............................                                  |  48%
  |                                                                       
  |................................                                 |  50%
  |                                                                       
  |..................................                               |  52%
  |                                                                       
  |...................................                              |  54%
  |                                                                       
  |....................................                             |  56%
  |                                                                       
  |.....................................                            |  57%
  |                                                                       
  |.......................................                          |  59%
  |                                                                       
  |........................................                         |  61%
  |                                                                       
  |.........................................                        |  63%
  |                                                                       
  |..........................................                       |  65%
  |                                                                       
  |...........................................                      |  67%
  |                                                                       
  |.............................................                    |  69%
  |                                                                       
  |..............................................                   |  70%
  |                                                                       
  |...............................................                  |  72%
  |                                                                       
  |................................................                 |  74%
  |                                                                       
  |.................................................                |  76%
  |                                                                       
  |...................................................              |  78%
  |                                                                       
  |....................................................             |  80%
  |                                                                       
  |.....................................................            |  81%
  |                                                                       
  |......................................................           |  83%
  |                                                                       
  |.......................................................          |  85%
  |                                                                       
  |.........................................................        |  87%
  |                                                                       
  |..........................................................       |  89%
  |                                                                       
  |...........................................................      |  91%
  |                                                                       
  |............................................................     |  93%
  |                                                                       
  |.............................................................    |  94%
  |                                                                       
  |...............................................................  |  96%
  |                                                                       
  |................................................................ |  98%
  |                                                                       
  |.................................................................| 100%

## output file: matrix.R

## [1] "matrix.R "