15.5 Cauchy-schwarz inequality

Introduction

Any treatment of linear algebra as relates to signal processing would not be complete without a discussion of the Cauchy-Schwarz ineqaulity, a relation that enables a wide array of signal procesing applications related to pattern matching through a method called the matched filter. Recall that in standard Euclidean space, the angle $\theta$ between two vectors $x,y$ is given by

Since $cos\left(\theta \right)\le 1$ , it follows that

Furthermore, equality holds if and only if $cos\left(\theta \right)=0$ , implying that

if and only if $y=ax$ for some real $a$ . This relation can be extended to all inner product spaces over a real or complex field and is known as the Cauchy-Schwarz inequality, which is of great importance to the study of signals.

The cauchy-schwarz inequality

Statement of the cauchy-schwarz inequality

The general statement of the Cauchy-Schwarz inequality mirrors the intuition for standard Euclidean space. Let $V$ be an inner product space over the field of complex numbers $\mathbb{C}$ with inner product $⟨·,·⟩$ . For every pair of vectors $x,y\in V$ the inequality

holds. Furthermore, the equality

holds if and only if $y=ax$ for some $a\in \mathbb{C}$ . That is, equality holds if and only if $x$ and $y$ are linearly dependent.

Proof of the cauchy-schwarz inequality

Let $V$ be a vector space over the real or complex field $F$ , and let $x,y\in V$ be given. In order to prove the Cauchy-Schwarz inequality, it will first be proven that ${\left|⟨x,y⟩\right|}^2=⟨x,x⟩⟨y,y⟩$ if $y=ax$ for some $a\in F$ . It will then be shown that ${\left|⟨x,y⟩\right|}^2<⟨x,x⟩⟨y,y⟩$ if $y\ne ax$ for all $a\in F$ .

Consider the case in which $y=ax$ for some $a\in F$ . From the properties of inner products, it is clear that

Hence, it follows that

Similarly, it is clear that

Thus, it is proven that ${\left|⟨x,y⟩\right|}^2=⟨x,x⟩⟨y,y⟩$ if $x=ay$ for some $a\in F$ .

Next, consider the case in which $y\ne ax$ for all $a\in F$ , which implies that $y\ne 0$ so $⟨y,y⟩\ne 0$ . Thus, it follows by the properties of inner products that, for all $a\in F$ , $⟨x-ay,x-ay⟩>0.$ This can be expanded using the properties of inner products to the expression

Choosing $a=\frac{⟨x,y⟩}{⟨y,y⟩}$ ,

Hence, it follows that $⟨x,x⟩-\frac{⟨x,y⟩⟨y,x⟩}{⟨y,y⟩}>0.$ Consequently, $⟨x,x⟩⟨y,y⟩-⟨x,y⟩\overline{⟨x,y}⟩>0.$ Thus, it can be concluded that ${\left|⟨x,y⟩\right|}^2<⟨x,x⟩⟨y,y⟩$ if $y\ne ax$ for all $a\in F$ .

Therefore, the inequality

holds for all $x,y\in V$ , and equality

holds if and only if $y=ax$ for some $a\in F$ .

Additional mathematical implications

Consider the maximization of $\left|〈\frac{x}{\left|\right|x\left|\right|},,,\frac{y}{\left|\right|y\left|\right|}〉\right|$ where the norm $\left|\right|·\left|\right|=⟨·,·⟩$ is induced by the inner product. By the Cauchy-Schwarz inequality, we know that ${\left|〈\frac{x}{\left|\right|x\left|\right|},,,\frac{y}{\left|\right|y\left|\right|}〉\right|}^2\le 1$ and that ${\left|〈\frac{x}{\left|\right|x\left|\right|},,,\frac{y}{\left|\right|y\left|\right|}〉\right|}^2=1$ if and only if $\frac{y}{\left|\right|y\left|\right|}=a\frac{x}{\left|\right|x\left|\right|}$ for some $a\in \mathbb{C}$ . Hence, $\left|〈\frac{x}{\left|\right|x\left|\right|},,,\frac{y}{\left|\right|y\left|\right|}〉\right|$ attains a maximum where $\frac{y}{\left|\right|y\left|\right|}=a\frac{x}{\left|\right|x\left|\right|}$ for some $a\in \mathbb{C}$ . Thus, collecting the scalar variables, $\left|〈\frac{x}{\left|\right|x\left|\right|},,,\frac{y}{\left|\right|y\left|\right|}〉\right|$ attains a maximum where $y=ax$ . This result will be particulaly useful in developing the matched filter detector techniques.