Matrices

Matrices can be used to express linear relations between variables. For example when we change coordinate systems from eg. \((x_1,x_2,x_3)\) to \((u_1,u_2,u_3)\) where $$ \left\{ \begin{align} u_1 &= 2x_1+3x_2+3x_3 \nonumber \\ u_2 &= 2x_1+4x_2+5x_3 \nonumber \\ u_3 &= x_1+x_2+2x_3 \nonumber \end{align} \right. \label{eq:linear} $$

Expressed as matrix product $$ \begin{align*} \underbrace{ \left[ \begin{matrix} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{matrix} \right] }_{A}\; \underbrace{ \left[ \begin{matrix} x_1 \\ x_2 \\ x_3 \end{matrix} \right] }_{X} &= \underbrace{ \left[ \begin{matrix} u_1 \\ u_2 \\ u_3 \end{matrix} \right] }_{U} \ A X &= U \end{align*} $$ Here \(A\) is a \(3\times 3\) matrix, and \(X\) is a vector or a \(3\times 1\) matrix.

Matrix Multiplication

Definition:

The entries in \(A X\) are the dot-product between the rows in \(A\) and the columns in \(X\), as shown below

For example, the entries of \(AB\) are $$ \left[ \begin{matrix} 1 & 2 & 3 & 4 \\ \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot \end{matrix} \right]\; \left[ \begin{matrix} 0 & \cdot \\ 3 & \cdot \\ 0 & \cdot \\ 2 & \cdot \end{matrix} \right] = \left[ \begin{matrix} 14 & \cdot \\ \cdot & \cdot \\ \cdot & \cdot \end{matrix} \right] $$

Properties:

• The width of \(A\) must equal the height of \(B\).
• The product \(AB\) has the same height as \(A\) and the same width as \(B\).
• Product \(AB\) represents: do transformation \(B\), then transformation \(A\). Unfortunately, you multiply from right to left. Similar to \(f(g(x))\), where you first apply \(g\) and then \(f\). The product \(BA\) is not even be defined when the width of \(B\) is not equal to the height of \(A\). In other words \(AB\ne BA\)
• They are well behaved associative products: \((AB)X =A(BX)\)
• \(BX\) means we apply transformation \(B\) to \(X\).

Identity matrix

Definition:

The identify matrix is a matrix that does no transformation: \(IX=X\)

The height of \(I\) needs to match the width of \(X\). \(I\) has \(1\)’s on the diagonal, and \(0\)’s everywhere else. $$ I_{n\times n} = \left[ \begin{matrix} 1 & & & \ldots & 0 \\ & 1 & & & \vdots \\ & & 1 & & \\ \vdots & & & \ddots & \\ 0 & \ldots & & & 1 \end{matrix} \right] \nonumber $$

For example: $$ I_{3\times3} = \left[ \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} \right] \nonumber $$

Rotation

Matrix \(R\), gives a \(\frac{\pi}{2}\) rotation. $$ R = \left[ \begin{matrix} 0 & -1 \\ 1 & 0 \end{matrix} \right] \nonumber $$

In general $$ R \left[ \begin{matrix} x \\ y \end{matrix} \right] = \left[ \begin{array}{r} -y \\ x \end{array} \right] \nonumber $$

Try multiplying with unity vector \(\hat\imath\), \(\hat\jmath\), or take \(R\) squared $$ \begin{align*} R\; \hat\imath &= \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{matrix} 1 \\ 0 \end{matrix} \right] = \left[ \begin{matrix} 0 \\ 1 \end{matrix} \right] = \hat\jmath \\ R\;\hat\jmath &= \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{matrix} 0 \\ 1 \end{matrix} \right] = \left[ \begin{array}{r} -1 \\ 0 \end{array} \right] = -\hat\imath \\ R^2 &= \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] = \left[ \begin{array}{rr} -1 & 0 \\ 0 & -1 \end{array} \right] = -I_{2\times 2} \end{align*} \nonumber $$

Inverse Matrix

Definition:

The inverse of matrix \(A\) is \(A^{-1}\) such that $$ \shaded{ \left\{ \begin{align*} A\;A^{-1} &= I \\ A^{-1}\;A &= I \end{align*} \right. } \nonumber $$

Referring to the system of equations \(\eqref{eq:linear}\), to express variables \(u_i\) in terms of \(x_i\) values, we need to inverse the transformation. For instance: in \(AX=B\); let matrix \(A\) and \(B\) be known what is \(X\)? $$ \begin{align*} AX &= B \Rightarrow \\ A^{-1}(AX) &= A^{-1} B \Rightarrow \\ IX &= A^{-1} B \Rightarrow \\ X &= A^{-1} B \end{align*} $$

Method

The inverse matrix is calculated using the adjoined matrix

$$ A^{-1}=\frac{1}{\mathrm{det}(A)}\;\mathrm{adj}(A) \nonumber $$

For this \(3\times 3\) example $$ A=\left[ \begin{matrix} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{matrix} \right] $$

First, find the determinant of \(A\) $$ \det(A)= \left| \begin{array}{rrr} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{array} \right| = 3 \nonumber $$

Second, find the minors (matrix of determinants) of matrix \(A\) $$ \mathrm{minors} = \left[\begin{array}{rrr} \left|\begin{array}{rrr} 4 & 5 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 5 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 4 \\ 1 & 1 \end{array}\right| \\ \left|\begin{array}{rrr} 3 & 3 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 1 & 2 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 1 & 1 \end{array}\right| \\ \left|\begin{array}{rrr} 3 & 3 \\ 4 & 5 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 2 & 5 \end{array}\right| & \left|\begin{array}{rrr} 2 & 3 \\ 2 & 4 \end{array}\right| \end{array}\right] = \left[\begin{array}{rrr} 3 & -1 & -2 \\ 3 & 1 & -1 \\ 3 & 4 & 2 \end{array}\right] \nonumber $$

Third, find the cofactors. Flip the signs checker board $$ \begin{array}{rrr} + & – & + \\ – & + & – \\ + & – & + \end{array} \nonumber $$ A ‘\(+\)’ means leave it alone. A ‘\(-\)’ means flip the sign. Apply the cofactors to the minors. $$ \left[\begin{array}{rrr} 3 & 1 & -2 \\ -3 & 1 & 1 \\ 3 & -4 & 2 \end{array}\right] \nonumber $$

Fourth, transpose (switch rows and columns) to find the adjoined matrix \(\mathrm{adj}(A)\). $$ \mathrm{adj}(A) = \left[\begin{array}{rrr} 3 & -3 & 3 \\ 1 & 1 & -4 \\ -2 & 1 & 2 \end{array}\right] \nonumber $$

The inverse matrix \(A^{-1}\) follows as $$ A^{-1} = \frac{1}{\det(A)}\;\mathrm{adj}(A) = \frac{1}{3} \left[\begin{array}{rrr} 3 & -3 & 3 \\ 1 & 1 & -4 \\ -2 & 1 & 2 \end{array}\right] \nonumber $$

Equations of planes

An equation of the form \(ax+by+cz=d\), expresses the condition for the point \((x,y,z)\) to be in the plane. It defines a plane.

Examples

Plane through the origin

Find the equation of the plane through the origin with normal vector \(\vec N = \left\langle 1, 5, 10 \right\rangle\).

Point \(P=(x,y,z)\) is in the plane when \(\vec{OP}\perp\vec{N}\). Therefore, their dot-product must equal zero (see vectors). $$ \begin{align*} \overrightarrow{OP}\cdot\vec{N} = 0 \\ \Leftrightarrow \left\langle x, y, z \right\rangle \cdot \left\langle 1, 5, 10 \right\rangle = 0 \\ \Leftrightarrow x + 5y + 10z = 0 \end{align*} $$

Plane not through the origin

Find the equation of the plane through \(P_0=(2,1,-1)\) with normal vector \(\vec N = \left\langle 1, 5, 10 \right\rangle\).

The normal vector is the same as in the first example, therefore it will be the same plane, but shifted so that it passes through \(P_0\).

Point \(P=(x,y,z)\) is in the plane when \(\overrightarrow{P_0P}\perp\overrightarrow{N}\). Therefore, their dot-product must equal zero (see vectors). This vector \(\overrightarrow{P_0P}\) equals \(P-P_0\). $$ \begin{align*} \left\langle x-2, y-1, z+1 \right\rangle \cdot \left\langle 1, 5, 10 \right\rangle &= 0 \\ \Leftrightarrow (x-2)+5(y-1)+10(z+1) &= 0 \\ \Leftrightarrow \underline{1}x+\underline{5}y+\underline{10}z &= -3 \end{align*} $$

In the equation \(ax+by+cz=d\), the coefficients \(\left\langle a,b,c\right\rangle\) is the normal vector \(\vec{N}\). Constant \(d\) indicates how far the plane is from the origin.

How could we have found the \(-3\) more quickly?

The first part of the equation is based on the normal vector $$ x + 5y + 10z = d \label{eq:planeequations2a} $$

We know \(P_0\) is in the plane. Substituting \(\left\langle x,y,z\right\rangle=P_0\) in \(\eqref{eq:planeequations2a}\) $$ \begin{align*} 1(2)+5(1)+10(-1) &= d \\ \Leftrightarrow d &= -3 \end{align*} \nonumber $$

Parallel or perpendicular?

Are vector \(\vec{v}=\left\langle 1,2,-1 \right\rangle\) and plane \(x+y+3z=5\) parallel, perpendicular or neither?

Vector \(\vec{v}\) is perpendicular to the plane when \(\vec{v}\)=\(s\;\vec{N}\), where \(s\) is a scalar. The normal vector follows from the coefficients of the plane equation $$ \vec{N} = \left\langle 1,1,3 \right\rangle \nonumber $$ Therefore \(\vec{V}\) is not perpendicular to the plane.

If \(\vec{v}\) is perpendicular to \(\vec{N}\), it is parallel to the plane. \(\vec{v}\perp\vec{N}\) when the dot-product equals zero. (see vectors) $$ \begin{align*} \vec{v}\cdot\vec{N} &= \left\langle 2, 1, -1 \right\rangle \cdot \left\langle 1, 1, 3 \right\rangle \\ &= 1+2-3 = 0 \end{align*} $$ Therefore, \(\vec{v}\) is parallel to the plane.

Solving systems of equations

To solve a system of equations, you try to find a point that is on several planes at the same time.

Example

Find the \(x,y,z\) that satisfies the conditions of the \(3\times 3\) linear system: $$ \left\{ \begin{align*} x+ z = 1 \\ x + y = 2 \\ x + 2y + 3z = 3 \end{align*} \right. $$

The first 2 equations represent two planes that intersect in line \(P_1\cap P_2\). The third plane intersects that line at the point \(P(x,y,z)\), the solution to the linear system.

Unless:

• if the line \(P_1\cap P_2\) is contained in \(P_3\), there are infinite many solutions. (Any point on the line is a solution.)
• if the line \(P_1\cap P_2\) is parallel to \(P_3\), then there are no solutions.

  

In matrix notation $$ \underbrace{ \left[\begin{array}{rrr} 1 & 0 & 1 \\ 1 & 1 & 0 \\ 1 & 2 & 3 \end{array}\right] }_{A}\; \underbrace{ \left[\begin{array}{ccc} x \\ y \\ z \end{array}\right] }_{X} = \underbrace{ \left[\begin{array}{rrr} 1 \\ 2 \\ 3 \end{array}\right] }_{B} \nonumber $$

The solution to \(AX=B\) is given by (see Inverse matrix) $$ X = A^{-1}B \nonumber $$

Recall

$$ A^{-1}=\frac{1}{\det (A)}\mathrm{adj}(A) \nonumber $$

This implies that matrix \(A\) is only invertible when $$ \shaded{ \det (A)\ne 0 } \nonumber $$

Theory

Homogeneous case

Homogeneous means that equations are invariant under scaling. In matrix notation: \(AX=0\).

For example: $$ \left\{ \begin{align*} x + z = 0 \\ x + y = 0 \\ x + 2y + 3z = 0 \end{align*} \right. $$

There is always the trivial solution: \((0,0,0)\).

Depending on the \(\det(A)\):

• If the \(\det (A)\ne 0\): \(A\) can be inverted. \(AX=0 \Leftrightarrow X=A^{-1}.0=0\). No other solutions.
• If the \(\det (A)= 0\): the determinant of \(\vec{N_1},\vec{N_2},\vec{N_3}\) equals \(0\). This implies that the plane’s normal vectors \(\vec{N_1}\), \(\vec{N_2}\) and \(\vec{N_3}\) are coplanar. A line through origin, perpendicular to plane of \(\vec{N_1}, \vec{N_2}, \vec{N_3}\) is parallel to all 3 planes and contained in them. Therefore there are infinite many solutions. To find the solutions, one can take the cross-product of two of the normals. It’s a nontrivial solution.

General case

The system $$ AX=B \nonumber $$

Depending on the \(\det(A)\)

• if the \(\det {A}\ne 0\): there is an unique solution \(X=A^{-1}B\)
• if the \(\det {A}=0\): either no solution, or infinitely many solutions. If you would solve it by hand and end up with \(0=0\), there are infinite solutions; if you end up with 1=2, there are no solutions.