\( \require{AMSsymbols}\def\lfz#1{\overset{\Large#1}{\,\circ\kern-6mu-\kern-7mu-\kern-7mu-\kern-6mu\bullet\,}} \def\laplace{\lfz{\mathscr{L}}} \def\fourier{\lfz{\mathcal{F}}} \def\ztransform{\lfz{\mathcal{Z}}} \require{cancel} \newcommand\ccancel[2][black] {\color{#1}{\cancel{\color{black}{#2}}}} \) The article on Z-Transforms introduced a difference equation for discrete stable causal Linear Time Invariant (LTI) systems, that from here on we will refer to as a LTI system, or system for short

$$ \begin{align} \sum_{k=0}^N a_k\,y[n-k] &= \sum_{k=0}^M b_k\,x[n-k]\quad\Rightarrow \nonumber \\[10mu] a_0y[n]+a_1y[n-1]+a_2y[n-2]+\ldots &= b_0x[n]+b_1x[n-1]+b_2x[n-2]+\ldots \nonumber \end{align} \nonumber $$

Here we will focus on the black box with discrete input signal \(x[n]\) and output \(y[n]\), where \(n\) is the sample number, sampled every \(T\) seconds.

We will following the notation from Z-transforms, where \(\ztransform\) is equivalent to the more common notation \(\mathfrak{Z}\left\{\,f[n]\,\right\}\), and \(f[n]\) is defined as the sample taken at time \(nT\). The terms filter and system will be used interchangeably.

## Linear Time Invariant System

The blackbox that we will examine is a *Stable Causal Linear Time Invariant* System (LTI). Most of the practical systems can be modeled as LTI systems or at least approximated by one around nominal operating point. Examples are include stereo engineering to counter the effect of a stadium on the music, or process control engineering in chemical plants.

The relation between the causal LTI system \(L\), its input \(x[n]\) and output \(y[n]\) can be expressed as $$ y[n]=\color{purple}{L\Big(\color{black}{x[n]}\Big)} \label{eq:ynLx} $$

### Definition

Before we start exploring the properties of stable causal LTI systems, let’s take a moment to define what a stable causal LTI system is.

A Causal system:

Is one in which changes in output do not precede changes in input.

The Linearity property states:

If input \(x_{1}(t)\), produces response \(L\big(x_{1}(t)\big)\), and input \(x_{2}(t)\), produces response \(L\big(x_{2}(t)\big)\), and \(a_i\) are real scalars, then a scaled and summed input produces the scaled and summed response$$ \color{purple}{L\Big(\color{green}{a_1}\color{blue}{x_1(t)}\color{black}{+}\color{green}{a_2}\color{blue}{x_2(t)}\Big)}=\color{green}{a_1} \color{purple}{L\Big(\color{blue}{x_1(t)}\Big)}+\color{green}{a_2} \color{purple}{L\Big(\color{blue}{x_2(t)}\Big)} \label{eq:linear} $$

The Time Invariance property states:

When we apply an input to the system now or \(\tau\) seconds from now, the output will be identical except for a time delay of \(a\) samples. That is, if the output due to input \(x(t)\) is \(y(t)\), then the output due to input \(x(t-\tau)\) is$$ \color{purple}{L\Big(\color{blue}{x(t-\tau)}\Big)} = \color{blue}{y(t-\tau)} \label{eq:timeinvariance} $$

Further more, the system needs to be stable:

LTI system is bounded-input bounded-output stable if all bounded inputs result in bounded outputs.

From here on we will refer to a stable causal linear time invariant system as a LTI system, or system for short.

### Combining systems

The two common configurations when combining filters are: series and parallel.

The figure below shows a *series*, or cascade, connection of filter \(H_1(z)\) and \(H_2(z)\), where the output from the first filter feeds the input of the next filter.

The transfer function for the series circuit is $$ H(z) = \frac{V(z)}{X(z)}\cdot\frac{Y(z)}{V(z)}= H_1(z)\,H_2(z) = H_2(z)\,H_1(z) $$ where the commutative property of multiplication implies that the order of the filters may be reversed.

The other common configuration is called *parallel* as shown below. In a parallel circuit, both filters get the same input signal and their outputs are summed.

The transfer function for the parallel circuit is $$ H(z) = \frac{Y_1(z)}{X(z)}+\frac{Y_2(z)}{X(z)}=H_1(z)+H_2(z)=H_2(z)+H_1(z) $$ where the commutative property of addition implies that the order of the filters may be reversed.

## Output of a Causal LTI system

In the time-domain, we find the output of a Causal LTI system system by passing the function of the input signal as a parameter to the system equation as shown in equation \(\eqref{eq:ynLx}\)

$$ y[n] = \color{purple}{L\Big(\color{black}{x[n]}\Big)} \nonumber $$

### Convolution in the time-domain

The article on Z-transforms showed how any discrete input signal \(x[n]\) can be expressed as a summation of scaled impulses.

$$ x[n] = \sum_{k=0}^{\infty}{\color{green}{x[k]}\ \color{blue}{\delta[n-k]}} \nonumber $$

Consider this \(x[n]\) to be the input to LTI system \(L\). The output of the system follows from substituting the equation for signal \(x[n]\) in \(\eqref{eq:ynLx}\) $$ y[n] = \color{purple}{L\Big(\color{black}{x[n]}\Big)}=\color{purple}{L\left(\color{black}{\sum_{k=0}^{\infty}\color{green}{x[k]}\ \underbrace{\color{blue}{\delta[n-k]}}_{\text{variable}}}\right)} \label{eq:yn} $$

Equation \(\eqref{eq:yn}\) matches the general form of a LTI system with a scalar and variable as a function of time \(t=\color{blue}{n}T\). This implies that we can use the part of the linearity property \(\eqref{eq:linear}\) that states:

$$ \color{purple}{L\Big(\color{green}{a}\ \color{blue}{x[n]}\Big)}=\color{green}{a}\ \color{purple}{L\Big(\color{blue}{x[n]}\Big)} $$

Apply the linearity property to bring the time independent part out of the system function \(L\) in equation \(\eqref{eq:yn}\) $$ \begin{align} y[n]=\color{purple}{L}\left(x[n]\right)=&\color{purple}{L}\left(\sum_{k=0}^{\infty}{\color{green}{x[k]}\,\ \color{blue}{\delta[n-k]}}\right) \nonumber \\ &= \sum_{k=0}^{\infty}{x[k]\,\ \underbrace{\color{purple}L\Big(\delta[n-k]\Big)}_{\color{blue}{\text{?}}}} \label{eq:yn2} \end{align} $$

Let’s examine the expression \(L\big(\delta[n-k]\big)\). Suppose we know how \(L\) acts on one impulse function \(\delta[n]\), and define it as
$$
\shaded{
h[n]\triangleq L\Big(\delta[n]\Big)
}
\label{eq:hn}
$$
this, so called, *impulse response* \(h[n]\) fully describes any LTI system, like the difference equation coefficients. The \(h[n]\) is also called the *Transfer Function* of the system.

Applying the time invariance property \(\eqref{eq:timeinvariance}\) to \(\eqref{eq:hn}\) yields $$ L\Big(\delta[n-k]\Big)=h[n-k] \label{eq:ldelta} $$

Substituting \(\eqref{eq:ldelta}\) in \(\eqref{eq:yn2}\), gives the convolution sum for LTI systems $$ \shaded{ y[n] = \sum_{k=0}^{\infty}h[n-k]\,x[k]\triangleq x[n]\ast h[n]\triangleq (x\star h)[n] } $$ or written out $$ y[n] = h[0]x[n]+h[1]x[n-1]+h[2]x[n-2]+\ldots+h[n]x[0] $$

In summary, when we model the transfer function of a LTI black box as \(h[n]\), in the time-domain the output signal \(y[n]\) is a convolution ‘\(\ast\)’ of input \(x[n]\) and the system impulse response \(h[n]\).

### Multiplication in the \(z\)-domain

Likewise, in the \(z\)-domain, the transfer function fully describes how the output signal \(Y(z)\) responds to an arbitrary input signal \(X(z)\).

As we have seen in Z-Transforms, the convolution in the time-domain transforms to a multiplication in the \(z\)-domain.

$$ (f\ast g)[n]\,\color{grey}{\gamma[n]} \ztransform F(z)\,G(z) \nonumber $$

That implies that the output \(Y(z)\) is the result of the input signal \(X(z)\) multiplied with the impulse response \(H(z)\) of the filter. $$ \shaded{ Y(z)=X(z)\,H(z) } \label{eq:yhz} $$ This is very convenient because it lets one determine the system response without having to solve the convolution.

## Transfer function

Time to take a closer look at the transfer function of the LTI system. We will express the transfer function as a ratio of polynomials and show it in its factorized form.

The Z-Transforms article opened with a generic form of Linear Constant-Coefficient Difference Equation (LCCDE) that expresses the relation between input \(x[n]\) and output \(y[n]\)

$$ \begin{align} \sum_{k=0}^N a_k\,\color{blue}{y[n-k]} &= \sum_{k=0}^M b_k\,\color{blue}{x[n-k]} \label{eq:lccde} \end{align} $$

Recall the time delay property

$$ f[n-a]\, \color{grey}{\gamma[n} – a \color{grey}{]} \,\ztransform z^{-a}F(z) \nonumber $$

Apply the time delay property to transform both sides of equation \(\eqref{eq:lccde}\) to the \(z\)-domain $$ \begin{align} \sum_{k=0}^N a_k\,\color{blue}{z^{-k}\,Y(z)}&=\sum_{k=0}^M b_k\,\color{blue}{z^{-k}X(z)}\quad\Rightarrow \nonumber \\ \color{blue}{Y(z)}\sum_{k=0}^N a_k\,\color{blue}{z^{-k}}&=\color{blue}{X(z)}\sum_{k=0}^M b_k\,\color{blue}{z^{-k}} \end{align} $$

Bring \(X(z)\) and \(Y(z)\) to the left and the summations to the right, yields the generic form of the transfer function \(H(z)\) $$ \shaded{ H(z) =\frac{Y(z)}{X(z)} =\frac{\sum_{k=0}^{M}b_k\,z^{-k}}{\sum_{k=0}^{N}a_k\,z^{-k}} =\frac{b_0+b_1z^{-1}+b_2z^{-2}+\cdots+b_Mz^{-M}}{a_0+a_1z^{-1}+a_2z^{-2}+\cdots+a_Nz^{-N}} } \label{eq:tf_polynominal} $$ note that \(a_0\) is typically assigned the value \(1\). Also note that some authors write this equation using subtractions for the \(a_{1\ldots{\small M}}\)coefficients, and negate the coefficients.

The \(b_k\) coefficients are called feedforward coefficients, and the \(a_k\) coefficients are called feedback coefficients.

### Factorized form

It is often convenient to factor the polynomials of the transfer function \(\eqref{eq:tf_polynominal}\), and write the function in terms of those factors.

The first step towards a factorized form, is to rewrite \(H(z)\) in a standard from, so that the highest order term of the numerator and denominator are unity. $$ \begin{align} H(z) &=K\,\frac{\frac{b_0}{b_M}+\frac{b_1}{b_M}z^{-1}+\frac{b_2}{b_M}z^{-2}+\frac{b_3}{b_M}z^{-3}+\cdots+z^{-M}}{\frac{a_0}{a_N}+\frac{a_1}{a_N}z^{-1}+\frac{a_2}{a_N}z^{-2}+\frac{a_3}{a_N}z^{-3}+\cdots+z^{-N}}, &K=\frac{b_M}{a_N} \end{align} $$

Factorize the polynomials $$ \begin{align} H(z)=K\,\frac{N(z)}{D(z)} &= K\,\frac{(1-q_1z^{-1})(1-q_2z^{-1})\cdots(1-q_{\small M}z^{-1})}{(1-p_1z^{-1})(1-p_2z^{-1})\cdots(1-p_{\small N}z^{-1})}&\times z^{\small M-N}\frac{z^{\small M}}{z^{\small N}} \nonumber \\[14mu] &= K\,z^{\small M-N}\,\frac{\prod_{i=1}^M(z-q_i)}{\prod_{i=1}^N(z-p_i)},\quad \text{ where }K=\frac{b_M}{a_N}\label{eq:tf_prod} \end{align} $$ where the factor \(z^{\small M-N}\) appears when the number of poles is not equal to the number of zeros.

The \(q_i\)’s are the roots of the equation \(N(z)=0\) and are called the system *zeros*. The \(p_i\)’s are the roots of the equation \(D(z)=0\) and are defined as the system *poles*. The *filter order* equals the number of poles or zeros in the transfer function, whichever is greater.

If one would make a 3D plot with the \(z\)-plane being the base and \(|H(z)|\) on the vertical axis, then the poles will show as thin “poles” pointing up and the zeros will show as dips pointing down.

The (complex) poles and zeros are properties of the transfer function, and therefore of the difference equation. Together with the gain constant \(K\) and delay \(z^{-(\small N-M})\) give a complete description of the filter.

## Visualization

The article Z-transforms introduced the normalized angular frequency \(\omega T\) and the \(z\)-plane. Here we will visualize the poles, zeroes and their evaluation for complex variable \(z\).

### Poles and zeros

As we have seen in equation \(\eqref{eq:tf_prod}\), the factorized transfer function can be written as $$ H(z) = K \frac{\prod_{i=1}^M(z-q_i)}{\prod_{i=1}^N(z-p_i)} \label{eq:visualpoleszeros} $$

The system may be represented graphically by plotting the poles and zeros in the complex \(z\)-plane. In these so called *pole-zero plots*, it is customary to mark a zero location with a circle (\(\circ\)) and a pole location with a cross (\(\times\)).

### Evaluating Poles and zeros

We will run ahead of ourselves and describe how the poles and zeroes affect the system response, later we will come back to this subject and explore it further.

Each of the poles \((z-p_i)\) and zeroes \((z-q_i)\) have a unique contribution to the transfer function. To give you an insight, we will we evaluate the factor \((z-p_i)\), but the same applies to zeroes.
$$
\begin{align}
(z-p_i)&={\Large(}\Re(z)+j\,\Im(z){\Large)}-{\Large(}\Re(p_i)+j\,\Im(p_i){\Large)} \nonumber \\[6mu]
&= {\Large(}\Re(z)-\Re(p_i){\Large)}+j\,{\Large(}\Im(z)-\Im(p_i){\Large)}
\end{align}
$$ this factor can be visualized with a vector drawn from \(r\) to \(z\). Each of the factors in the numerator and denominator may be interpreted as a vector in the *z*-plane, originating from the zero \(z_i\) or pole \(p_i\) and directed to the point \(z\) at which the function is evaluated.

The length and angle of these factors represent their contribution to the transfer function. For example for a pole \(p_i=\Re(p_i)+j\,\Im(p_i)\), the magnitude and angle of the vector to the variable \(z=\Re(z)+j\,\Im(z)\) are $$ \begin{aligned} |z-p_i| &= \sqrt{{\Large(}\Re(z)-\Re(p_i){\Large)}^2+{\Large(}\Im(z)-\Im(p_i){\Large)}^2} \\[6mu] \angle(z-p_i) &= \mathrm{atan2}{\Large(}\Im(z)-\Im(p_i),\,\Re(z)-\Re(p_i){\Large)} \end{aligned} $$ where \(\mathrm{atan2}\) is defined as $$ \mathrm{atan2}(y,x)\triangleq \begin{cases} \arctan\left(\frac{y}{x}\right) & x \gt 0 \\ \arctan\left(\frac{y}{x}\right)+\pi & x \lt 0 \land y \geq 0 \\ \arctan\left(\frac{y}{x}\right)-\pi & x \lt 0 \land y \lt 0 \\ \frac{\pi}{2} & x= 0 \land y \gt 0 \\ -\frac{\pi}{2} & x= 0 \land y \lt 0 \\ \text{undefined} & x= 0 \land y = 0 \end{cases} $$

### Multiplication and division

While we are on the subject, recall multiplication and division of complex numbers as it is most easily done in polar form $$ \begin{align} Z_1Z_2& = |Z_1|e^{j\angle{Z_1}}\ |Z_2|e^{j\angle{Z_2}} = |Z_1||Z_2|e^{j(\angle{Z_1}+\angle{Z_2{)}}} &\text{multiplication} \\[10mu] \frac{Z_1}{Z_2} &= \frac{|Z_1|e^{j\angle{Z_1}}}{|Z_2|e^{j\angle{Z_2}}} = \frac{|Z_1|}{|Z_2|}e^{j(\angle{Z_1}-\angle{Z_2{)}}} &\text{division} \end{align} $$ Refer to Complex Arithmetic Formulas for an overview of various complex operations.

Applying \(|K|=K\) and \(\angle{K}=\mathrm{atan2}(0,K)=0\), the magnitude and angle of the complete transfer function \(H(z)\) may be written as $$ \shaded{ \begin{align} H(z) &= |H(z)|\ e^{j\angle{H(z)}} \nonumber \\[10mu] \text{where }\quad|H(z)| &= K \frac{\prod_{i=1}^m\left|(z-q_i)\right|}{\prod_{i=1}^n\left|(z-p_i)\right|} \nonumber \\[10mu] \text{and }\quad\angle{H(z)}&=\sum_{i=1}^m\angle(z-q_i)-\sum_{i=1}^n\angle(z-p_i) \nonumber \end{align} } $$

Suggested next reading is Evaluating Discrete Transfer Functions.