# Matrices



My notes of the excellent lectures of “Denis Auroux. 18.02 Multivariable Calculus. Fall 2007. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu. License: Creative Commons BY-NC-SA.”

Matrices can be used to express linear relations between variables. For example when we change coordinate systems from eg. $$(x_1,x_2,x_3)$$ to $$(u_1,u_2,u_3)$$ where \left\{ \begin{align} u_1 &= 2x_1+3x_2+3x_3 \nonumber \\ u_2 &= 2x_1+4x_2+5x_3 \nonumber \\ u_3 &= x_1+x_2+2x_3 \nonumber \end{align} \right. \label{eq:linear}

Expressed as matrix product \begin{align*} \underbrace{ \left[ \begin{matrix} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{matrix} \right] }_{A}\; \underbrace{ \left[ \begin{matrix} x_1 \\ x_2 \\ x_3 \end{matrix} \right] }_{X} &= \underbrace{ \left[ \begin{matrix} u_1 \\ u_2 \\ u_3 \end{matrix} \right] }_{U} \ A X &= U \end{align*} Here $$A$$ is a $$3\times 3$$ matrix, and $$X$$ is a vector or a $$3\times 1$$ matrix.

## Matrix Multiplication

Definition:

The entries in $$A X$$ are the dot-product between the rows in $$A$$ and the columns in $$X$$, as shown below

For example, the entries of $$AB$$ are $$\left[ \begin{matrix} 1 & 2 & 3 & 4 \\ \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \end{matrix} \right]\; \left[ \begin{matrix} 0 & \cdot \\ 3 & \cdot \\ 0 & \cdot \\ 2 & \cdot \end{matrix} \right] = \left[ \begin{matrix} 14 & \cdot \\ \cdot & \cdot \\ \cdot & \cdot \end{matrix} \right]$$

Properties:

• The width of $$A$$ must equal the height of $$B$$.
• The product $$AB$$ has the same height as $$A$$ and the same width as $$B$$.
• Product $$AB$$ represents: do transformation $$B$$, then transformation $$A$$. Unfortunately, you multiply from right to left. Similar to $$f(g(x))$$, where you first apply $$g$$ and then $$f$$. The product $$BA$$ is not even be defined when the width of $$B$$ is not equal to the height of $$A$$. In other words $$AB\ne BA$$
• They are well behaved associative products: $$(AB)X =A(BX)$$
• $$BX$$ means we apply transformation $$B$$ to $$X$$.

## Identity matrix

Definition:

The identify matrix is a matrix that does no transformation: $$IX=X$$

The height of $$I$$ needs to match the width of $$X$$. $$I$$ has $$1$$’s on the diagonal, and $$0$$’s everywhere else. $$I_{n\times n} = \left[ \begin{matrix} 1 & & & \ldots & 0 \\ & 1 & & & \vdots \\ & & 1 & & \\ \vdots & & & \ddots & \\ 0 & \ldots & & & 1 \end{matrix} \right] \nonumber$$

For example: $$I_{3\times3} = \left[ \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} \right] \nonumber$$

### Rotation

Matrix $$R$$, gives a $$\frac{\pi}{2}$$ rotation. $$R = \left[ \begin{matrix} 0 & -1 \\ 1 & 0 \end{matrix} \right] \nonumber$$

In general $$R \left[ \begin{matrix} x \\ y \end{matrix} \right] = \left[ \begin{array}{r} -y \\ x \end{array} \right] \nonumber$$

Try multiplying with unity vector $$\hat\imath$$, $$\hat\jmath$$, or take $$R$$ squared \begin{align*} R\; \hat\imath &= \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{matrix} 1 \\ 0 \end{matrix} \right] = \left[ \begin{matrix} 0 \\ 1 \end{matrix} \right] = \hat\jmath \\ R\;\hat\jmath &= \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{matrix} 0 \\ 1 \end{matrix} \right] = \left[ \begin{array}{r} -1 \\ 0 \end{array} \right] = -\hat\imath \\ R^2 &= \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] = \left[ \begin{array}{rr} -1 & 0 \\ 0 & -1 \end{array} \right] = -I_{2\times 2} \end{align*} \nonumber

## Inverse Matrix

Definition:

The inverse of matrix $$A$$ is $$A^{-1}$$ such that \shaded{ \left\{ \begin{align*} A\;A^{-1} &= I \\ A^{-1}\;A &= I \end{align*} \right. } \nonumber
That implies that $$A$$ must be a square matrix ($$n \times n$$).

Referring to the system of equations $$\eqref{eq:linear}$$, to express variables $$u_i$$ in terms of $$x_i$$ values, we need to inverse the transformation. For instance: in $$AX=B$$; let matrix $$A$$ and $$B$$ be known what is $$X$$? \begin{align*} AX &= B \Rightarrow \\ A^{-1}(AX) &= A^{-1} B \Rightarrow \\ IX &= A^{-1} B \Rightarrow \\ X &= A^{-1} B \end{align*}

### Method

The inverse matrix is calculated using the adjoined matrix

$$A^{-1}=\frac{1}{\mathrm{det}(A)}\;\mathrm{adj}(A) \nonumber$$

For this $$3\times 3$$ example $$A=\left[ \begin{matrix} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{matrix} \right]$$

First, find the determinant of $$A$$ $$\det(A)= \left| \begin{array}{rrr} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{array} \right| = 3 \nonumber$$

Second, find the minors (matrix of determinants) of matrix $$A$$ $$\mathrm{minors} = \left[\begin{array}{rrr} \left|\begin{array}{rrr} 4 & 5 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 5 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 4 \\ 1 & 1 \end{array}\right| \\ \left|\begin{array}{rrr} 3 & 3 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 1 & 1 \end{array}\right| \\ \left|\begin{array}{rrr} 3 & 3 \\ 4 & 5 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 2 & 5 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 2 & 4 \end{array}\right| \end{array}\right] = \left[\begin{array}{rrr} 3 & -1 & -2 \\ 3 & 1 & -1 \\ 3 & 4 & 2 \end{array}\right] \nonumber$$

Third, find the cofactors. Flip the signs checker board $$\begin{array}{rrr} + & – & + \\ – & + & – \\ + & – & + \end{array} \nonumber$$ A ‘$$+$$’ means leave it alone. A ‘$$-$$’ means flip the sign. Apply the cofactors to the minors. $$\left[\begin{array}{rrr} 3 & 1 & -2 \\ -3 & 1 & 1 \\ 3 & -4 & 2 \end{array}\right] \nonumber$$

Fourth, transpose (switch rows and columns) to find the adjoined matrix $$\mathrm{adj}(A)$$. $$\mathrm{adj}(A) = \left[\begin{array}{rrr} 3 & -3 & 3 \\ 1 & 1 & -4 \\ -2 & 1 & 2 \end{array}\right] \nonumber$$

The inverse matrix $$A^{-1}$$ follows as $$A^{-1} = \frac{1}{\det(A)}\;\mathrm{adj}(A) = \frac{1}{3} \left[\begin{array}{rrr} 3 & -3 & 3 \\ 1 & 1 & -4 \\ -2 & 1 & 2 \end{array}\right] \nonumber$$

## Equations of planes

An equation of the form $$ax+by+cz=d$$, expresses the condition for the point $$(x,y,z)$$ to be in the plane. It defines a plane.

### Examples

#### Plane through the origin

Find the equation of the plane through the origin with normal vector $$\vec N = \left\langle 1, 5, 10 \right\rangle$$.

Point $$P=(x,y,z)$$ is in the plane when $$\vec{OP}\perp\vec{N}$$. Therefore, their dot-product must equal zero (see vectors). \begin{align*} \overrightarrow{OP}\cdot\vec{N} = 0 \\ \Leftrightarrow \left\langle x, y, z \right\rangle \cdot \left\langle 1, 5, 10 \right\rangle = 0 \\ \Leftrightarrow x + 5y + 10z = 0 \end{align*}

#### Plane not through the origin

Find the equation of the plane through $$P_0=(2,1,-1)$$ with normal vector $$\vec N = \left\langle 1, 5, 10 \right\rangle$$.

The normal vector is the same as in the first example, therefore it will be the same plane, but shifted so that it passes through $$P_0$$.

Point $$P=(x,y,z)$$ is in the plane when $$\overrightarrow{P_0P}\perp\overrightarrow{N}$$. Therefore, their dot-product must equal zero (see vectors). This vector $$\overrightarrow{P_0P}$$ equals $$P-P_0$$. \begin{align*} \left\langle x-2, y-1, z+1 \right\rangle \cdot \left\langle 1, 5, 10 \right\rangle &= 0 \\ \Leftrightarrow (x-2)+5(y-1)+10(z+1) &= 0 \\ \Leftrightarrow \underline{1}x+\underline{5}y+\underline{10}z &= -3 \end{align*}

In the equation $$ax+by+cz=d$$, the coefficients $$\left\langle a,b,c\right\rangle$$ is the normal vector $$\vec{N}$$. Constant $$d$$ indicates how far the plane is from the origin.

##### How could we have found the $$-3$$ more quickly?

The first part of the equation is based on the normal vector $$x + 5y + 10z = d \label{eq:planeequations2a}$$

We know $$P_0$$ is in the plane. Substituting $$\left\langle x,y,z\right\rangle=P_0$$ in $$\eqref{eq:planeequations2a}$$ \begin{align*} 1(2)+5(1)+10(-1) &= d \\ \Leftrightarrow d &= -3 \end{align*} \nonumber

#### Parallel or perpendicular?

Are vector $$\vec{v}=\left\langle 1,2,-1 \right\rangle$$ and plane $$x+y+3z=5$$ parallel, perpendicular or neither?

Vector $$\vec{v}$$ is perpendicular to the plane when $$\vec{v}$$=$$s\;\vec{N}$$, where $$s$$ is a scalar. The normal vector follows from the coefficients of the plane equation $$\vec{N} = \left\langle 1,1,3 \right\rangle \nonumber$$ Therefore $$\vec{V}$$ is not perpendicular to the plane.

If $$\vec{v}$$ is perpendicular to $$\vec{N}$$, it is parallel to the plane. $$\vec{v}\perp\vec{N}$$ when the dot-product equals zero. (see vectors) \begin{align*} \vec{v}\cdot\vec{N} &= \left\langle 2, 1, -1 \right\rangle \cdot \left\langle 1, 1, 3 \right\rangle \\ &= 1+2-3 = 0 \end{align*} Therefore, $$\vec{v}$$ is parallel to the plane.

## Solving systems of equations

To solve a system of equations, you try to find a point that is on several planes at the same time.

### Example

Find the $$x,y,z$$ that satisfies the conditions of the $$3\times 3$$ linear system: \left\{ \begin{align*} x+ z = 1 \\ x + y = 2 \\ x + 2y + 3z = 3 \end{align*} \right.

The first 2 equations represent two planes that intersect in line $$P_1\cap P_2$$. The third plane intersects that line at the point $$P(x,y,z)$$, the solution to the linear system.

Unless:

• if the line $$P_1\cap P_2$$ is contained in $$P_3$$, there are infinite many solutions. (Any point on the line is a solution.)
• if the line $$P_1\cap P_2$$ is parallel to $$P_3$$, then there are no solutions.

In matrix notation $$\underbrace{ \left[\begin{array}{rrr} 1 & 0 & 1 \\ 1 & 1 & 0 \\ 1 & 2 & 3 \end{array}\right] }_{A}\; \underbrace{ \left[\begin{array}{ccc} x \\ y \\ z \end{array}\right] }_{X} = \underbrace{ \left[\begin{array}{rrr} 1 \\ 2 \\ 3 \end{array}\right] }_{B} \nonumber$$

The solution to $$AX=B$$ is given by (see Inverse matrix) $$X = A^{-1}B \nonumber$$

Recall

$$A^{-1}=\frac{1}{\det (A)}\mathrm{adj}(A) \nonumber$$

This implies that matrix $$A$$ is only invertible when $$\shaded{ \det (A)\ne 0 } \nonumber$$

### Theory

#### Homogeneous case

Homogeneous means that equations are invariant under scaling. In matrix notation: $$AX=0$$.

For example: \left\{ \begin{align*} x + z = 0 \\ x + y = 0 \\ x + 2y + 3z = 0 \end{align*} \right.

There is always the trivial solution: $$(0,0,0)$$.

Depending on the $$\det(A)$$:

• If the $$\det (A)\ne 0$$: $$A$$ can be inverted. $$AX=0 \Leftrightarrow X=A^{-1}.0=0$$. No other solutions.
• If the $$\det (A)= 0$$: the determinant of $$\vec{N_1},\vec{N_2},\vec{N_3}$$ equals $$0$$. This implies that the plane’s normal vectors $$\vec{N_1}$$, $$\vec{N_2}$$ and $$\vec{N_3}$$ are coplanar. A line through origin, perpendicular to plane of $$\vec{N_1}, \vec{N_2}, \vec{N_3}$$ is parallel to all 3 planes and contained in them. Therefore there are infinite many solutions. To find the solutions, one can take the cross-product of two of the normals. It’s a nontrivial solution.

### General case

The system $$AX=B \nonumber$$

Depending on the $$\det(A)$$

• if the $$\det {A}\ne 0$$: there is an unique solution $$X=A^{-1}B$$
• if the $$\det {A}=0$$: either no solution, or infinitely many solutions. If you would solve it by hand and end up with $$0=0$$, there are infinite solutions; if you end up with 1=2, there are no solutions.