6.3 Inner product space

Next we equip a normed vector space $V$ with a notion of "direction".

• An inner product is a function ( $:(·\cdot ·, V\times V)\to \mathbb{C}$ ) such that the following properties hold ( $\forall x, y, z, x\in V\land y\in V\land z\in V$ and $\forall \alpha , \alpha \in $ ):
  • $x\cdot y=\overline{y\cdot x}$
  • $x\cdot \alpha y=\alpha (x\cdot y)$ ...implying that $\alpha x\cdot y=\overline{\alpha }(x\cdot y)$
  • $x\cdot y+z=x\cdot y+x\cdot z$
  • $x\cdot x\ge 0$ with equality iff $x=0$
  In simple terms, the inner product measures the relativealignment between two vectors. Adding an inner product operation to a vector space yields an inner product space . Important examples include:
  • $V=\mathbb{R}^N$ , $≔(x\cdot y, x^Ty)$
  • $V=\mathbb{C}^N$ , $≔(x\cdot y, (x)y)$
  • $V=l_2$ , $≔(\{x(n)\}\cdot \{y(n)\}, \sum_{n=()} )$∞ ∞ x n y n
  • $V={ℒ}_2$ , $≔(f(t)\cdot g(t), \int_{()} \,d t)$∞ ∞ f t g t

The inner products above are the "usual" choices for those spaces.

The inner product naturally defines a norm: $$≔((x), \sqrt{x\cdot x})$$ though not every norm can be defined from an inner product.

• The Cauchy-Schwarz inequality says

$$\left|x\cdot y\right|\le (x)(y)$$ with equality iff $\exists , \alpha \in \colon x=\alpha y$ .

When $(x\cdot y)\in \mathbb{R}$ , the inner product can be used to define an "angle" between vectors: $$\cos \theta =\frac{x\cdot y}{(x)(y)}$$

• Vectors $x$ and $y$ are said to be orthogonal , denoted as $\perp (x, y)$ , when $x\cdot y=0$ . The Pythagorean theorem says: $$\forall , \perp (x, y)\colon (x+y)^2=(x)^2+(y)^2$$ Vectors $x$ and $y$ are said to be orthonormal when $\perp (x, y)$ and $(x)=(y)=1$ .
• $\perp (x, S)$ means $\perp (x, y)$ for all $y\in S$ . $S$ is an orthogonal set if $\perp (x, y)$ for all $(x\land y)\in S$ s.t. $x\neq y$ . An orthogonal set $S$ is an orthonormal set if $(x)=1$ for all $x\in S$ . Some examples of orthonormal sets are
  • $\mathbb{R}^3$ : $S=\{\begin{pmatrix}1\\ 0\\ 0\\ \end{pmatrix}, \begin{pmatrix}0\\ 1\\ 0\\ \end{pmatrix}\}$
  • $\mathbb{C}^N$ : Subsets of columns from unitary matrices
  • $l_2$ : Subsets of shifted Kronecker delta functions $S\subset \{\{\delta (n-k)\}\colon k\in \mathbb{Z}\}$
  • ${ℒ}_2$ : $S=\{\frac{1}{\sqrt{T}}f(t-nT)\colon n\in \mathbb{Z}\}$ for unit pulse $f(t)=u(t)-u(t-T)$ , unit step $u(t)$
  where in each case we assume the usual inner product.