1.3 Vector spaces

Vector spaces

Definition 1 A linear vector space $(X,R,+,·)$ is given by a signal space $X$ (called vectors), a set of scalars $R$ , an addition operation $+:X\times X\to X$ , and a multiplication operation $·:R\times X\to X$ , such that:

1. $X$ forms a group under addition:
   1. $\forall x,y\in X\exists !x+y\in X,$ (closed under addition)
   2. $\exists 0\in X$ such that $0+X=X+0=X.$ (additive identity)
   3. $\forall x\in X\exists y\in X$ such that $x+y=0,$ (additive inverse)
   4. $\forall x,y,z\in Xx+(y+z)=(x+y)+z.$ (associative law)
2. Multiplication has the following properties: for any $x,y\in X$ and $a,b\in R$ :
   1. $a·x\in X,$ (closure in $X$ under multiplication)
   2. $a·(b·x)=(a·b)·x,$ (compatibility)
   3. $(a+b)·x=a·x+b·x,$ (distributive law over $R$ )
   4. $a·(x+y)=a·x+a·y.$ (distributive law over $X$ )
3. The set $R$ has the following properties:
   1. There exists $1\in R$ s.t. $1·x=x\forall x\in X,$ (multiplication identity)
   2. There exists $0\in R$ s.t. $0·x=0\forall x\in X.$ (multiplicative null element)

Example 1 Here are some examples of vector spaces:

• $X={\mathbb{R}}^n$ (space of all vectors of length $n$ ) over $R=\mathbb{R}$ is a vector space.
• $X={\mathbb{C}}^n$ ( $\mathbb{C}$ is complex numbers) over $R=\mathbb{C}$ is a vector space.
• $X={\mathbb{R}}^n$ over $R=\mathbb{C}$ is not a vector space, because closure in $X$ under multiplication is not met.
• $X=C\left[T\right]$ (continuous functions in $T$ ) over $R=\mathbb{R}$ is a vector space.

Subspaces

Definition 2 A subset $M\subseteq X$ is a linear subspace of $X$ if $M$ itself is a linear vector space. Note that, in particular, this implies that any subspace $M$ must obey $0\in M$ .

Example 2 Here are some examples of subspaces:

• In $X={\mathbb{R}}^2$ over $R=\mathbb{R}$ , any line that passes through the origin is a subspace of $X$ :
  $$M=\left\{(x,y),\in ,{\mathbb{R}}^2,,,\text{such},,\text{that},,,\frac{y}{x},=,c\right\}.$$
• In $X=C\left[T\right]$ over $R=\mathbb{R}$ , the followings are subspaces of $X$ :
  $$\begin{array}{cc}\hfill M_1& =\{f\left(x\right)=a{x}^2+bx+c:a,b,c\in \mathbb{R}\}\hfill \\ \hfill M_2& =\{f\left(x\right):f\left(x_0\right)=0\}.\hfill \end{array}$$
  In contrast, the set $M_3=\{f\left(x\right):f\left(x_0\right)=a\ne 0\}$ is not a subspace.

Proposition 1 If $M$ and $N$ are subspaces of $X$ , then $M\cap N$ is also a subspace.

Proof: We assume that $M$ and $N$ hold properties of linear vector space, and show that so does $M\cap N$ :

1. $$x,y\in M\cap N\Rightarrow \left\{\begin{array}{c}x,y\in M\Rightarrow x+y\in M\\ x,y\in N\Rightarrow x+y\in N\end{array}\right\}\Rightarrow x+y\in M\cap N$$
2. $$\left\{\begin{array}{c}M\text{linear}\text{vector}\text{space}\Rightarrow 0\in M\\ N\text{linear}\text{vector}\text{space}\Rightarrow 0\in N\end{array}\right\}\Rightarrow 0\in M\cap N$$
3. $$x\in M\cap N\Rightarrow \left\{\begin{array}{c}x\in M\exists y\in M\text{s.t.}x+y=0\\ x\in N\exists y\in N\text{s.t.}x+y=0\end{array}\right\}\Rightarrow y\in M\cap N$$

The other properties are shown in a similar fashion.

Definition 3 A vector $x\in X$ , where $(X,R,+,·)$ is a vector space, is a linear combination of a set $\{x_1,x_2,...,x_n\}\subseteq X$ if it can be written as $x={\sum }_{i=1}^n{a}_i·x_i$ , $a_i\in R$ . The set of all linear combinations of a set of points $\{x_1,x_2,...,x_n\}$ builds a linear subspace of $X$ .

Example 3 $Q={\sum }_{i=0}^2{a}_i{x}^i$ is a linear subspace of $\left(C\right[T],\mathbb{R},+,·)$ containing the set of all quadratic functions, as it corresponds to all linear combinations of the set of functions $\{x^2,x,1\}$ .

Bases and spans

Definition 4 For the set $S=\{x_1,x_2,...,x_n\}\subseteq X$ , the span of $S$ is written as

Example 4 The space of quadratic functions $Q$ is written as $Q=\left[S_1\right]$ , with $S_1=\{x^2,x,1\}$ . The space can also be written as $\left[S_2\right]$ with $S_2=\{1,x,x^2-2\})$ , i.e. $\left[S_1\right]=\left[S_2\right]$ . To prove this, we need to show $\left[S_2\right]\subseteq \left[S_1\right]$ and $\left[S_1\right]\subseteq \left[S_2\right]$ . For the former case we have

which means that every element that can be spanned by $S_2$ , can also be spanned by $S_1$ , and hence $\left[S_2\right]\subseteq \left[S_1\right]$ . The latter case can be shown in a similar manner.

Definition 5 A set $S$ is a linearly independent set if

Otherwise, the set $S$ is linearly dependent .

Definition 6 A finite set $S$ of linearly independent vectors is a basis for the space $X$ if $\left[S\right]=X$ , i.e. if $X$ is spanned by $S$ .

Definition 7 The dimension of $X$ is the number of elements of its basis $\left|S\right|$ . A vector space for which a finite basis does not exist is called an infinite-dimensional space .

Theorem 1 Any two bases of a subspace have the same number of elements.

Proof: We prove by contradiction: assume that $S_1=\{x_1,...,x_n\}$ and $S_2=\{y_1,...,y_m\}$ , $m>n$ , are two bases for a subspace $X$ with different numbers of elements. We have that since $y_1\in X$ it can be written as a linear combination of $S_1$ :

Order the elements of $S_1$ above so that $a_1$ is nonzero; since $y_1$ must be nonzero then at least one such $a_i$ must exist. Solving the above equation for $x_1$ yields

Thus $\{y_1,x_2,x_3,...,x_n\}$ is a basis, in terms of which we can write any vector of the space $X$ , including $y_2$ :

Since $y_1,y_2$ are linearly independent, at least one of the values of $b_i,i>2,$ must be nonzero. Sort the remaining $x_i$ so that $b_2$ is nonzero. Solving for $x_2$ results in

Therefore, $\{y_1,y_2,x_3,...,x_n\}$ is a basis for $X$ . Continuing in this way, we can eliminate each $x_i$ , showing that $\{y_1,y_2,...,y_n\}$ is a basis for $X$ . Thus, we have $y_{n+1}={\sum }_{i=1}^n{c}_i{y}_i$ , or equivalently:

As a result, $S_2$ is linearly dependent and is not a basis. Therefore, all bases of $X$ must have the same number of elements.

Basis representations

Having a basis in hand for a given subspace allows us to express the points in the subspace in more than one way. For each point $x\in \left[S\right]$ in the span of a basis $S=\{S_1,S_2...S_n\}$ , that is,

there is a one-to-one map (i.e., an equivalence) between $x\in \left[S\right]$ and $a=\{a_1,...,a_n\}\in R^n$ , that is, both $x$ and $a$ uniquely identify the point in $S$ . This is stated more formally as a theorem.

Theorem 2 If $S$ is a linearly independent set, then

if and only if $a_i=b_i$ for $i=1,2...n$ .

Proof: Theorem 1 states that the scalars $\{a_1,...,a_n\}$ are unique for $x$ . We begin by assuming that indeed

This implies

Since the elements of $S$ are linearly independent, each one of the scalars of the sum must be zero, that is, $a_i-b_i=0$ and so $a_i=b_i$ for each $i=1,...,n$ .

Example 5 (Digital Communications) A transmitter sends two waveforms:

The signal $r\left(t\right)$ recorded by the receiver is continuous, that is, $r\left(t\right)\in C\left[T\right]$ . Assuming that the propagation delay is known and corrected at the receiver, we will have they the received signal must be in the span of the possible transmitted signals, i.e., $r\left(t\right)\in span(S_1\left(t\right),S_0\left(t\right))$ . One can check that $S_1\left(t\right)$ and $S_2\left(t\right)$ are linearly independent. Thus, one can use a unique choice of coefficients $a_0$ and $a_1$ that denote whether bit 0 or bit 1 is transmitted and contain the amount of attenuation caused by the transmission:

The uniqueness of this representation can only be obtained if the transmitted signals $S_0\left(t\right)$ and $S_1\left(t\right)$ are linearly independent. The waveforms above are used in in phase shift keying (PSK); other similar examples include frequency shift keying (FSK) and quadrature amplitude modulation (QAM).