Let's see what the eigenvectors of $H$ are. These will be the finite-length signals that, when input into the system, emerge as outputs simply as scaled versions of themselves. In that sense, they are somehow fundamentally related to all LTI systems.
It so happens that, remarkably, any and all LTI systems for finite-length (length $N$) signals have the exact same set of eigenvectors! The eigenvectors for any LTI length-$N$ system are complex harmonic sinusoids:
$s_k[n]~=~ \frac{e^{j \frac{2\pi}{N}kn}}{\sqrt{N}} ~=~ \frac{1}{\sqrt{N}}\left( \cos\!\left(\frac{2\pi}{N}kn\right) + j \sin\!\left(\frac{2\pi}{N}kn\right) \right),
\qquad 0\leq n,k \leq N-1$So, if we have an LTI system--any LTI system--then giving an $s_k$ as an input will result in the output being $\lambda_k s_k$, with the particular values of $\lambda_k$ of course being dependent on the system:
When a complex harmonic sinusoid is input into an LTI system, the output is a scaled version of the input. Here the real (cosine) and imaginary (sine) parts of the sinusoid are plotted as the input. Note how the system merely scales the inputs. To prove this special property of LTI systems, we simply compute the circular convolution sum for an LTI system with arbitrary impulse response $h[n]$ and input of the form $\frac{e^{j \frac{2\pi}{N}kn}}{\sqrt{N}}$:$\begin{align*}
s_k[n]\circledast h[n]&=
\sum_{m=0}^{N-1} s_k[(n-m)_N]\,h[m]\\&= \sum_{m=0}^{N-1} \frac{e^{j \frac{2\pi}{N}k(n-m)_N}}{\sqrt N} \,h[m] \\&=\sum_{m=0}^{N-1} \frac{e^{j \frac{2\pi}{N}k(n-m)}}{\sqrt N} \,h[m]\\&=\sum_{m=0}^{N-1} \frac{e^{j \frac{2\pi}{N}kn}}{\sqrt N} e^{-j \frac{2\pi}{N}km} \, h[m] \\&=\left( \sum_{m=0}^{N-1} e^{-j \frac{2\pi}{N}km} \,h[m] \right)\frac{e^{j \frac{2\pi}{N}kn}}{\sqrt N}\\&=~ \lambda_k \, s_k[n]~,~\lambda_k=\left( \sum_{m=0}^{N-1} e^{-j \frac{2\pi}{N}km} \,h[m]\right)
\end{align*}$
This proof reveals how we are to find the eigenvalues ($\lambda_k$) that correspond to each harmonic sinusoid eigenvector, they are simply the inner products of the eigenvectors with the system's impulse response. Each value $\lambda_k$ is called the system's
frequency response at frequency $k$, because it indicates how the system scales inputs of that particular frequency. It is a significant enough characteristic of the system to warrant its own notation: $H[k]$.
Eigendecomposition of lti systems
As with matrices in general, we can apply an eigendecomposition on an LTI system matrix. For LTI finite-length systems, these matrices are circulant:
CAPTION. We have seen that the eigenvectors of LTI systems are harmonic complex sinusoids. We can stack these up into a single matrix $S$, the entries of which are $S_{n,k}$, which is plotted below:
The real part of the eigenvector matrix $S$: $\cos(\frac{2\pi}{N}kn)/\sqrt{N}$.The real part of the eigenvector matrix $S$: $\sin(\frac{2\pi}{N}kn)/\sqrt{N}$.Graphical representation of the real and imaginary parts of the discrete-time finite length LTI system eigenvector matrix $S$. Likewise, we can plot the respective eigenvalues of the eigenvectors, which above we defined to be the values of the frequency response of the system:
$\Lambda = \begin{bmatrix} \lambda_0 \\&\lambda_1 \\&&\ddots \\&&&\lambda_{N-1} \end{bmatrix}
~=~ \begin{bmatrix} H_u[0]\\&H_u[1] \\&&\ddots \\&&&H_u[N-1] \end{bmatrix}$Putting the equation $y=Hx$ together with the decomposition $H=S\Lambda S^H$, we have:
Eigendecomposition of discrete-time finite length LTI systems. We already know one way of understanding how LTI systems operate on signal inputs: they convolve them with the system's impulse response (represented in linear algebra form by the equation $y=Hx$, where $H$ is circulant). The eigendecomposition gives us another understanding. The matrix $S^H$ takes the input and extracts what would be the coefficients of the input's representation as a linear combination of harmonic sinusoids (it turns out this is called the discrete Fourier transform). Then, multiplication by the diagonal matrix $\Lambda$ modifies these coefficients in a way that is particular to the system $H$ (for ALL LTI systems have the same $S$ and $S^H$). Finally, multiplication with the matrix $S$ takes the modified coefficients and expresses them as a linear combination of harmonic sinusoids to give the output $y$ (it turns out that operation is called in inverse discrete Fourier transform).
