1.3 Review of linear algebra

Vector spaces are the principal object of study in linear algebra. A vector space is always defined with respectto a field of scalars.

Fields

A field is a set $F$ equipped with two operations, addition and mulitplication, and containing two special members 0 and 1( $0\neq 1$ ), such that for all $\{a, b, c\}\in F$

• • $(a+b)\in F$
  • $a+b=b+a$
  • $(a+b)+c=a+(b+c)$
  • $a+0=a$
  • there exists $-a$ such that $a+-a=0$
• • $ab\in F$
  • $ab=ba$
  • $abc=abc$
  • $a1=a$
  • there exists $a^{(-1)}$ such that $aa^{(-1)}=1$
• $a(b+c)=ab+ac$

• $F$ is an abelian group under addition
• $F$ is an abelian group under multiplication
• multiplication distributes over addition

Examples

,,

Vector spaces

Let $F$ be a field, and $V$ a set. We say $V$ is a vector space over $F$ if there exist two operations, defined for all $a\in F$ , $u\in V$ and $v\in V$ :

• vector addition: ( $u$ , $v$ ) $(u+v)\in V$
• scalar multiplication: ( $a$ , $v$ ) $av\in V$

• • $u+(v+w)=(u+v)+w$
  • $u+v=v+u$
  • $u+0=u$
  • there exists $-u$ such that $u+-u=0$
• • $a(u+v)=au+av$
  • $(a+b)u=au+bu$
  • $abu=abu$
  • $1u=u$

• $V$ is an abelian group under plus
• Natural properties of scalar multiplication

Examples

• $\mathbb{R}^N$ is a vector space over
• $\mathbb{C}^N$ is a vector space over
• $\mathbb{C}^N$ is a vector space over
• $\mathbb{R}^N$ is not a vector space over

Euclidean space

Throughout this course we will think of a signal as a vector $$x=\left(\begin{array}{c}{x}_1\\ x_2\\ \\ x_N\end{array}\right)=\begin{pmatrix}{x}_1 & x_2 & & x_N\\ \end{pmatrix}^T$$ The samples $\{x_i\}$ could be samples from a finite duration, continuous time signal, for example.

A signal will belong to one of two vector spaces:

Real euclidean space

$x\in \mathbb{R}^N$ (over)

Complex euclidean space

$x\in \mathbb{C}^N$ (over)

Subspaces

Let $V$ be a vector space over $F$ .

A subset $S\subseteq V$ is called a subspace of $V$ if $S$ is a vector space over $F$ in its own right.

$S\subseteq V$ is a subspace if and only if for all $a\in F$ and $b\in F$ and for all $s\in S$ and $t\in S$ , $(as+bt)\in S$

Linear independence

Let $u_1,,u_k\in V$ .

We say that these vectors are linearly dependent if there exist scalars $a_1,,a_k\in F$ such that

If only holds for the case $a_1==a_k=0$ , we say that the vectors are linearly independent .

Spanning sets

Consider the subset $S=\{v_1, v_2, , v_k\}$ . Define the span of $S$ $$<S>\equiv \mathrm{span}(S)\equiv \{\sum_{i=1}^k a_i{v}_i\colon a_i\in F\}$$

Fact: $<S>$ is a subspace of $V$ .

Aside

If $S$ is infinite, the notions of linear independence and span are easily generalized:

We say $S$ is linearly independent if, for every finite collection $u_1,,u_k\in S$ , ( $k$ arbitrary) we have $$(\sum_{i=1}^k a_i{u}_i=0)\implies \forall i\colon a_i=0$$ The span of $S$ is $$<S>=\{\sum_{i=1}^k a_i{u}_i\colon a_i\in F\land u_i\in S\land (k)\}$$∞

Bases

A set $B\subseteq V$ is called a basis for $V$ over $F$ if and only if

• $B$ is linearly independent
• $<B>=V$

Key fact

If $B$ is a basis for $V$ , then every $v\in V$ can be written uniquely (up to order of terms) in the form $$v=\sum_{i=1}^N a_i{v}_i$$ where $a_i\in F$ and $v_i\in B$ .

Other facts

• If $S$ is a linearly independent set, then $S$ can be extended to a basis.
• If $<S>=V$ , then $S$ contains a basis.