Review of linear algebra

Dimension

Let $V$ be a vector space with basis $B$ . The dimension of $V$ , denoted $\dim (V)$ , is the cardinality of $B$ .

Every vector space has a basis.

Every basis for a vector space has the same cardinality.

$\implies \dim (V)$ is well-defined .

If $\dim (V)$∞ , we say $V$ is finite dimensional .

Examples

Every subspace is a vector space, and therefore has its own dimension.

Facts

• If $S$ is a subspace of $V$ , then $\dim (S)\le \dim (V)$ .
• If $\dim (S)=\dim (V)$∞ , then $S=V$ .

Direct sums

Let $V$ be a vector space, and let $S\subseteq V$ and $T\subseteq V$ be subspaces.

We say $V$ is the direct sum of $S$ and $T$ , written $V=(S, T)$ , if and only if for every $v\in V$ , there exist unique $s\in S$ and $t\in T$ such that $v=s+t$ .

If $V=(S, T)$ , then $T$ is called a complement of $S$ .

Facts

• Every subspace has a complement
• $V=(S, T)$ if and only if
  • $S\cap T=\{0\}$
  • $<S,T>=V$
• If $V=(S, T)$ , and $\dim (V)$∞ , then $\dim (V)=\dim (S)+\dim (T)$

Proofs

Invoke a basis.

Norms

Let $V$ be a vector space over $F$ . A norm is a mapping $(V, F)$ , denoted by $()$ , such that forall $u\in V$ , $v\in V$ , and $\in F$

• $(u)> 0$ if $u\neq 0$
• $(u)=\left|\right|(u)$
• $(u+v)\le (u)+(v)$

Examples

Euclidean norms:

$x\in \mathbb{R}^N$ : $$(x)=\sum_{i=1}^N x_i^2^{\left(\frac{1}{2}\right)}$$ $x\in \mathbb{C}^N$ : $$(x)=\sum_{i=1}^N \left|x_i\right|^2^{\left(\frac{1}{2}\right)}$$

Induced metric

Every norm induces a metric on $V$ $$d(u, v)\equiv (u-v)$$ which leads to a notion of "distance" between vectors.

Inner products

Let $V$ be a vector space over $F$ , $F=\mathbb{R}$ or $\mathbb{C}$ . An inner product is a mapping $V\times VF$ , denoted $\cdot$ , such that

• $v\cdot v\ge 0$ , and $(v\cdot v=0, v=0)$
• $u\cdot v=\overline{v\cdot u}$
• $au+bv\cdot w=a(u\cdot w)+b(v\cdot w)$

Examples

$\mathbb{R}^N$ over: $$x\cdot y=x^Ty=\sum_{i=1}^N x_i{y}_i$$

$\mathbb{C}^N$ over: $$x\cdot y=(x)y=\sum_{i=1}^N \overline{x_i}{y}_i$$

If $(x=\left(\begin{array}{c}{x}_1\\ \\ x_N\end{array}\right))\in \mathbb{C}$ , then $$(x)\equiv \left(\begin{array}{c}\overline{x_1}\\ \\ \overline{x_N}\end{array}\right)^T$$ is called the "Hermitian," or "conjugatetranspose" of $x$ .

Triangle inequality

If we define $(u)=u\cdot u$ , then $$(u+v)\le (u)+(v)$$ Hence, every inner product induces a norm.

Cauchy-schwarz inequality

For all $u\in V$ , $v\in V$ , $$\left|u\cdot v\right|\le (u)(v)$$ In inner product spaces, we have a notion of the angle between two vectors: $$((u, v)=\arccos \left(\frac{u\cdot v}{(u)(v)}\right))\in \left[0 , 2\pi \right)$$

Orthogonality

$u$ and $v$ are orthogonal if $$u\cdot v=0$$ Notation: $(u, v)$ .

If in addition $(u)=(v)=1$ , we say $u$ and $v$ are orthonormal .

In an orthogonal (orthonormal) set , each pair of vectors is orthogonal (orthonormal).

Orthonormal bases

An Orthonormal basis is a basis $\{v_i\}$ such that $$v_i\cdot v_i={}_{ij}=\begin{cases}1 & \text{if $i=j$}\\ 0 & \text{if $i\neq j$}\end{cases}$$

Expansion coefficients

If the representation of $v$ with respect to $\{v_i\}$ is $$v=\sum a_i{v}_i$$ then $$a_i=v_i\cdot v$$

Gram-schmidt

Every inner product space has an orthonormal basis. Any (countable) basis can be made orthogonal by theGram-Schmidt orthogonalization process.

Orthogonal compliments

Let $S\subseteq V$ be a subspace. The orthogonal compliment $S$ is $$S^{}=\{u\colon u\in V\land (u\cdot v=0)\land \forall v\colon v\in S\}$$ $S^{}$ is easily seen to be a subspace.

If $\dim (v)$∞ , then $V=(S, S^{})$ .

Linear transformations

Loosely speaking, a linear transformation is a mapping from one vector space to another that preserves vector space operations.

More precisely, let $V$ , $W$ be vector spaces over the same field $F$ . A linear transformation is a mapping $T:VW$ such that $$T(au+bv)=aT(u)+bT(v)$$ for all $a\in F$ , $b\in F$ and $u\in V$ , $v\in V$ .

In this class we will be concerned with linear transformations between (real or complex) Euclidean spaces , or subspaces thereof.

Image

$$()$$ T w w W T v w for some v

Nullspace

Also known as the kernel: $$\ker (T)=\{v\colon v\in V\land (T(v)=0)\}$$

Both the image and the nullspace are easily seen to be subspaces.

Rank

$$\mathrm{rank}(T)=\dim (())$$ T

Nullity

$$\mathrm{null}(T)=\dim (\ker (T))$$

Rank plus nullity theorem

$$\mathrm{rank}(T)+\mathrm{null}(T)=\dim (V)$$

Matrices

Every linear transformation $T$ has a matrix representation . If $T:{𝔼}^N{𝔼}^M$ , $𝔼=\mathbb{R}$ or $\mathbb{C}$ , then $T$ is represented by an $M\times N$ matrix $$A=\begin{pmatrix}{a}_{11} & & a_{1N}\\ & & \\ a_{M1} & & a_{MN}\\ \end{pmatrix}$$ where $\left(\begin{array}{c}{a}_{1i}\\ \\ a_{Mi}\end{array}\right)=T(e_i)$ and $e_i=\left(\begin{array}{c}0\\ \\ 1\\ \\ 0\end{array}\right)$ is the $i^{\mathrm{th}}$ standard basis vector.

Column span

$$\mathrm{colspan}(A)=<A>=()$$ A

Duality

If $A:{\mathbb{R}}^N{\mathbb{R}}^M$ , then $$\ker (A)^{}=()$$ A

If $A:{\mathbb{C}}^N{\mathbb{C}}^M$ , then $$\ker (A)^{}=()$$ A

Inverses

The linear transformation/matrix $A$ is invertible if and only if there exists a matrix $B$ such that $AB=BA=I$ (identity).

Only square matrices can be invertible.

Let $A:{𝔽}^N{𝔽}^N$ be linear, $𝔽=\mathbb{R}$ or $\mathbb{C}$ . The following are equivalent:

• $A$ is invertible (nonsingular)
• $\mathrm{rank}(A)=N$
• $\mathrm{null}(A)=0$
• $\det A\neq 0$
• The columns of $A$ form a basis.

If $A^{(-1)}=A^T$ (or $(A)$ in the complex case), we say $A$ is orthogonal (or unitary ).