2.2 The strength of a vector

In the study of signal processing, the "strength" of signals (which, remember, can be expressed as vectors) is often of interest. It is not immediately clear what standard could be used for judging the strength of a vector. A similar problem comes up when determining the strength of a person: does it have to do with the amount of weight that can be lifted, and/or the number of times it can be lifted, and if so what kind of weight or lift? So it is with vectors; consider the two below:

Norms: the strength of vectors

The Euclidean length , or $2$-norm , of a vector $x\in C^N$ is defined as: $\|x\|_2 = \sqrt{ \sum_{n=0}^{N-1} |x[n]|^2 }$ This norm, like all norms, is a function that takes a vector as its argument and then produces a real number that is always greater than or equal to 0. The 2-norm is a very common norm, so often it will be seen without even having a subscript, i.e. as $\|x\|$.

Let's calculate a 2-norm. Suppose: $x=\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$Then $\|x\|_2$, the 2-norm of $x$, is: $\| x \|_2 = \sqrt{ \sum_{n=0}^{N-1} |x[n]|^2} = \sqrt{1^2 + 2^2 + 3^2} = \sqrt{14}$

It happens that the 2-norm is actually just one kind of a more general type of norm, called $p$-norms . For any real number $p\geq 1$, the $p$-norm is: $\|x\|_p = \left( \sum_{n=0}^{N-1} |x[n]|^p \right)^{1/p}$ So the 2-norm is a $p$-norm with $p=2$. Another norm of interest to us is when $p=1$, the 1-norm : $\|x\|_1 = \sum_{n=0}^{N-1} |x[n]|$

A final norm that we will consider is called the $\infty$-norm . It is essentially what happens as the $p$ in the $p$-norm increases to $\infty$, but has this easy to calculate expression: $\|x\|_\infty=\max_n |x[n]|$ So if you have a vector $x$ and take the absolute value of all the entries, the $\infty$ norm of $x$ is simply the largest of those values.

The meaning of different norms

Or consider another example of the utility of different norms. Suppose you are designing a computer-driven car. You clearly will want to make sure it does not deviate from the programmed path. Call the deviation at a given time the error signal $d$. Would you seek to minimize $\|d\|_2$ or $\|d\|_infty$? If you minimize $\|d\|_2$, then the car will, overall, keep to the path very closely, but quick and large deviations would be permitted: if over the course of a long drive the car followed the path perfectly, except for a fraction of a second it accidentally swerved into the other lane, the $\|d\|_2$ would be small, but that could potentially still be a disaster! Instead, you would want to minimize the $\|d\|_infty$ value, so the car never moves more than a maximum distance from the path.

Normalizing a vector

If you have studied signal processing, then you will know that you can consider each track as a signal, and that each track will have a different norm. In order to make the loudness the same, you would like to modify each track so that it has the same "strength", or norm (technically, unless all the tracks are the same length, you would like them to have the same norm per time length). Thankfully, it is very straightforward to modify a signal so as to change its norm. There are two steps.

First, you will modify the vector so that it is a unit norm, i.e. a norm of 1. This is called normalizing the vector. To do that, you simply scale the vector by its norm. Suppose your vector is $x$; the normalized version of $x$ is then $\frac{x}{\|x\|_2}$. For example, earlier we saw that the vector: $x=\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$had a norm of $\sqrt{14}$. The normalized version of $x$ then is $x$ divided by that value: $x=\begin{bmatrix} \frac{1}{\sqrt{14}} \\ \frac{2}{\sqrt{14}} \\ \frac{3}{\sqrt{14}} \end{bmatrix}$That new vector has a 2-norm of 1, as desired. If you wanted the norm of the vector to be some other value, say the number $M$, you would multiply the normalized vector by $M$.