Linear transformations

Contents

• Linear Transformations
• Examples from geometry
• Examples from algebra
• Examples from differential equations
• Null space and range; rank
• Solving linear equations
  • Does $Tx=y$ have a solution?
  • What is the form of the general solution to $Tx=y$?
• Systems of equations
  • More equations than unknowns, i.e. $m\gt n$
  • More unknowns than equations, i.e. $m\lt n$
  • As many equations as unknowns, i.e. $m = n$
• The components of a vector with respect to a basis
• Matrix representation of a linear transformation
• Special case: $A:\F^n\to\F^m$
• Composition of transformations and matrix multiplication
• The vector space $\mathcal{L}(V,W)$
• Inverses and other powers of a linear transformation
  • Solving linear equations

Linear Transformations

Definition. A map $T:V\to W$ from one vector space $V$ to another $W$ is called linear if for all $x, y\in V$ and all $a, b\in\mathbb{F}$ one has $T(ax+by) = aT(x)+bT(y)$.

Synonym: a linear map is also called a linear transformation or a linear operator.

In this definition it is assumed that $V$ and $W$ are vector spaces over the same field $\mathbb{F}$. If the two vector spaces are defined over different fields then the concept of linear map from $V$ to $W$ is not defined.

Sometimes it is more convenient to split the task of verifying that a certain linear map $T:V\to W$ is linear into two steps:

• Verify that $T(x+y) = T(x)+T(y)$ for all $x, y\in V$
• Verify that $T(ax)=a T(x)$ for all $x\in V$ and $a\in\mathbb{F}$.

Problem. Show that the definition implies that if $T:V\to W$ is linear, then $T(0_V)=0_W$.

Examples from geometry

Rotations in the plane. Let $V=W=\R^2$ and $\mathbb{F}=\R$ and consider the map $R:\R^2\to\R^2$ defined by counterclockwise rotation by an angle of $\theta$ radians. Show that $R$ is linear. Find a formula for $R\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}$.

Reflections in the plane. Again consider $V=W=\R^2$, and let $\ell\subset\R^2$ be some line through the origin. Define $S(x)$ to be the reflection of $x$ in the line $\ell$.

Show that $S:\R^2\to\R^2$ is linear. and find a formula for $S\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}$, in the case where $\ell$ is the diagonal $x_1=x_2$.

Projection onto a line. Let $V=\R^2$ and $W$ be a line through the origin in $\R^2$. Consider the map $P:V\to W$ for which $Px$ is the orthogonal projection onto $W$. Show that $P$ is a linear transformation.

Rigid rotation in $\R^3$. In this example, let $V=W=\R^3$, and let $T:\R^3\to\R^3$ be the map defined by first rotating around the $z$-axis over $30^\circ$ and then rotating around the $x$-axis over $45^\circ$. Show that $T:\R^3\to\R^3$ is linear.

We postpone finding a formula for $T\left(\begin{smallmatrix} x_1 \\ x_2 \\x_3 \end{smallmatrix}\right)$ to the next chapter on matrices.

Examples from algebra

Let $a,b,c,d\in\mathbb{F}$ be given numbers in the field $\mathbb{F}$ and consider the map $T:\mathbb{F}^2 \to \mathbb{F}^2$ given by

$T\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} =\begin{pmatrix} ax_1+bx_2 \\ cx_1+dx_2 \end{pmatrix}$

for all $\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}\in\mathbb{F}^2$. Show that $T$ is linear. Visualize the map $T$ you get if $a=2$, $b=c=0$, $d=\frac12$.

Examples from differential equations

Let $W=\mathcal{F}(\R,\R)$, and let $V\subset W$ be the subspace of functions that are differentiable. Consider the map $D:V\to W$ given by

$(Df)(x) = f'(x).$

Thus $D(e^x) = e^x$, $D(\sin x) = \cos x$, $D(x^2)=2x$, etc.

Verify that $D:V\to W$ is linear.

Null space and range; rank

Definition. If $T:V\to W$ is a linear map, then

• the Null space of $T$ is $N(T) = \{x\in V \mid T(x)=0\}$
• the Range of $T$ is $R(T) = \{T(x) \mid x\in V\} = \{y\in W \mid \exists x\in V: y=Tx\}$.

The null space is sometimes also called the kernel of the map and the notation $N(T) = \mathrm{ker}\,T$ is sometimes used.

Theorem. The Null space a linear transformation $T:V\to W$ is linear subspace of $V$; the range of $T$ is a linear subspace of $W$.

Definition. The rank of $T$ is the dimension of the range of $T$.

Injectivity Theorem. A linear map $T:V\to W$ is injective if and only if $N(T)=\{0\}$.

Rank+Nullity Theorem. If $T:V\to W$ is linear, and if $V$ is finite dimensional, then

Injectivity Theorem. If $T:V\to W$ is a linear transformation then $T$ is injective if and only if $N(T)=\{0\}$.

Bijectivity Theorem. If $V$ and $W$ are finite dimensional vector spaces with the same dimension, and if $T:V\to W$ is a linear transformation then the following are equivalent:

A very important special case is when $V=W$ and $V$ is finite dimensional.

Solving linear equations

Let $T:V\to W$ be a linear transformation between vector spaces, and consider the equation

$Tx=y.$

Here $y\in W$ is given and $x\in V$ is the unknown. The standard questions are is there a solution? and if there is a solution, how many? Linear algebra provides the following answers:

Does $Tx=y$ have a solution?

$Tx=y$ has a solution if and only if $y\in R(T)$. This is the definition of the range of a linear transformation. What does this tell us? It doesn't say we can always solve $Tx=y$, but the set of $y$ for which there are solutions has a nice property — $R(T)$ is a linear subspace of $W$. Therefore, if we can find a solution to $Tx_1=y_1$ and $Tx_2=y_2$ then we can also find a solution to $Tz=c_1y_1+c_2y_2$ (one solution is $z=c_1x_1+c_2x_2$ — there might be others).

What is the form of the general solution to $Tx=y$?

Suppose that $x, x'\in V$ both are solutions, i.e. $T(x)=T(x')=y$. Then $T(x-x')=0$, so $x-x'\in N(T)$: so we see that the difference between any two solutions lies in the null space of $T$.

Conversely, suppose that $x$ is a solution, i.e. $T(x)=y$, and suppose $u\in N(T)$. Then $T(x+u)=T(x)+T(u)=y+0=y$, i.e. $x+u$ is also a solution. Hence, given a solution $x$ to $T(x)=y$ one can get another solution by adding any vector $u$ in the null space to $x$

Particular solutions and the homogeneous equation. If $x_p\in V$ is a solution of $T(x)=y$, then every solution of $T(x)=y$ is given by

$x = x_p+x_h\quad\text{with }x_h\in N(T).$

In this context the following terminology is very often used:

• $T(x)=y$ is the inhomogenous equation
• $T(x)=0$ is the homogeneous equation
• $x_p$ is a particular solution
• $x_h\in N(T)$ the general solution of the homogeneous equation

If $r\stackrel{\rm def}{=}\dim N(T)<\infty$, and if we know a basis $\{u_1, \dots, u_r\}$ for the null space $N(T)$, then every vector $x_h$ in the null space is given by

$x_h = c_1u_1+\cdots+c_ru_r$

for certain $c_1, \dots, c_r\in\mathbb{F}$. If we still know a particular solution $x_p$ of $T(x)=y$ then the general solution to the equation $T(x)=y$ (i.e. every solution) is given by

$x = x_p+c_1u_1+\cdots+c_ru_r$

where $c_1, \dots, c_r\in\mathbb{F}$ are arbitrary constants.

Systems of equations

Consider the linear transformation $A:\mathbb{F}^n\to\mathbb{F}^m$ given by

$A\begin{bmatrix} x_1\\ \vdots \\x_n \end{bmatrix} \stackrel{\rm def}{=} \begin{bmatrix} a_{11}x_1+\cdots+a_{1n}x_n \\ \vdots \\ a_{m1}x_1+\cdots+a_{mn}x_n \end{bmatrix}$

For any vector $y = \left[\begin{smallmatrix} y_1\\\vdots\\y_m \end{smallmatrix}\right] \in \mathbb{F}^m$ the equation $Ax=y$ is then equivalent with the system of linear equations for the unknowns $x_1, \dots, x_n$ given by

$\begin{aligned} a_{11}x_1 + \cdots + a_{1n}x_n \,&= y_1 \\ \vdots\quad& \\ a_{m1}x_1 + \cdots + a_{mn}x_n &= y_m \end{aligned}$

In other words, we are considering $m$ linear equations with $n$ unknowns.

In this setting the rank+nullity theorem says that $\dim N(A)+\dim R(A)=\dim V$, i.e. $\dim N(A) + \dim R(A) = n$.

More equations than unknowns, i.e. $m\gt n$

It follows from $\dim N(A) + \dim R(A) = n$ that $\dim R(A) = n-\dim N(A) \leq n \lt m$. So in this case $R(A)$ is always a proper subspace of $W$: the equation $T(x)=y$ does not have a solution for most $y$.

For those $y\in \mathbb{F}^m$ for which the system does have a solution the general solution contains $r$ constants, where $r=\dim N(A) = n-\dim R(A)$.

More unknowns than equations, i.e. $m\lt n$

Since $R(A)\subset \R^m$, we have $\dim R(A)\leq m$. By the nullity+rank theorem,

$\dim N(A) = \dim V-\dim R(A) = n-\dim R(A)\geq n-m \gt 0.$

So in this case the dimension of the null space always is positive, i.e. there are nonzero solutions to $Ax=0$. For any $y\in \mathbb{F}^m$ one of the following occurs:

• $y\not\in R(A)$ and there is no solution
• $y\in R(A)$ and the general solution contains $r$ free constants, where $r=\dim N(A)\gt 0$.

As many equations as unknowns, i.e. $m = n$

We again have $\dim N(A) = n-\dim R(A)$.

If $A$ is injective then $N(A)=\{0\}$, and hence $\dim N(A)=0$, so that $\dim R(A) = n$. Since $R(A)\subset \R^n$, this implies that when $A$ is injective, $A$ also is surjective.

If on the other hand $A$ is not injective then $\dim N(A)>0$ and thus $\dim R(A)<n$. In this situation $R(A)$ is a proper subspace of $\R^n$ and therefore $Ax=y$ does not have a solution for all $y$.

The components of a vector with respect to a basis

Definition. An ordered basis of a vector space $V$ is an ordered list of vectors $\beta = (v_1, \dots, v_n)$ such that $\{v_1, \dots, v_n\}$ is a basis of $V$.

If $\beta = (v_1, \dots, v_n)$ is an ordered basis of $V$ and $x\in V$ is any vector then there exist $x_1, \dots, x_n\in \mathbb{F}$ such that

$x = x_1v_1 + \cdots + x_nv_n \,.$

The numbers $x_1, \dots, x_n$ are called the components of $x$ with respect to the basis $\beta$. These components determine a column vector. In the notation of the text book:

$[x]_\beta= \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix} \in\mathbb{F}^n.$

Instead of components, the $x_i$ are sometimes also called the coordinates of $x$.

Matrix representation of a linear transformation

Let $T:V\to W$ be a linear transformation, and let $\beta = \{v_1, \dots, v_n\}$ be an ordered basis for $V$ and $\gamma = \{w_1, \dots, w_m\}$ an ordered basis for $W$. Each vector $Tv_i$ can be written as a linear combination of $w_1, \dots, w_m$, i.e. there exist numbers $a_{ij}\in\mathbb{F}$ such that

$Tv_i = a_{1i}w_1 + \cdots + a_{mi}w_m \qquad (i=1, 2, \dots, n).$

Linearity of $T$ allows us to compute $T(x)$ if we know the coefficients $a_{ij}$ and the components $x_j$ of $x$ in the basis $v_1, \dots, v_n$. Namely one has:

$\begin{aligned} Tx &= T(x_1v_1+\cdots +x_nv_n) \\ &=x_1 T(v_1) + \cdots + x_n T(v_n) \\ &=x_1 \left(a_{11}w_1 + \cdots + a_{m1}w_m\right) + \cdots + x_n \left(a_{1n}w_1 + \cdots + a_{mn}w_m\right) \\ &\qquad\text{(rearrange terms)} \\ &=\left(a_{11}x_1+\cdots+a_{1n}x_n\right) w_1 + \cdots + \left(a_{m1}x_1+\cdots+a_{mn}x_n\right) w_m \end{aligned}$

Thus

$[Tx]_\gamma = \begin{pmatrix} a_{11}x_1+\cdots+a_{1n}x_n \\ \vdots \\ a_{m1}x_1+\cdots+a_{mn}x_n \end{pmatrix}$

The coefficients $a_{ij}$ of the linear transformation $T$ with respect to the ordered bases $\beta$ and $\gamma$ form a matrix

$[T]_\beta^\gamma = \begin{pmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{m1} & \cdots & a_{mn} \end{pmatrix}$

which is called the matrix representation of $T$ in the bases $\beta, \gamma$.

Since it turns out to be easy to confuse rows and columns the following observation may be helpful: the first column \tmat a_{11} \\ \cdot \\ \cdot \\ a_{m1} \trix of the matrix $[T]_\beta^\gamma$ contains the components of $Tv_1$ expressed in the basis $\{w_1, \dots, w_n\}$.

Example. If $m=3$ and $n=2$, so that $W$ is three dimensional with basis $\gamma=\{w_1, w_2, w_3\}$ and $V$ is two dimensional with basis $\beta=\{v_1, v_2\}$, and if

$Tv_1 = w_1-3w_2+5w_3, \qquad Tv_2=-w_1,$

then the matrix of $T$ in these bases is

$[T]_\beta^\gamma = \begin{pmatrix} 1 & -1 \\ -3 & 0 \\ 5 & 0 \end{pmatrix}.$

Special case: $A:\F^n\to\F^m$

The vector space $\F^n$ has the standard basis $\{e_1, \dots, e_n\}$. If the matrix of a linear transformation $A:\F^m\to\F^n$ is given by $(a_{ij})$, then $A$ is given by matrix multiplication

$Ax = \begin{pmatrix} a_{11} & \cdots & a_{1n} \\ \vdots && \vdots \\ a_{m1} & \cdots & a_{mn} \end{pmatrix} \begin{pmatrix} x_1 \\ \vdots \\ x_m \end{pmatrix}$

Composition of transformations and matrix multiplication

If we have three vector spaces $U, V, W$ and linear transformations $A:V\to W$ and $B:U\to V$ then we can define the composition $AB:U\to W$ by

$(AB)(x) \stackrel{\rm def}{=} A\bigl(B(x)\bigr) \text{ for all $x\in U$}.$

Theorem. If $A:V\to W$ and $B:U\to V$ are linear transformations of vector spaces $U,V,W$, then the composition $AB:U\to W$ is also linear.

The proof is a homework problem.

If we have ordered bases $\alpha=\{u_1, \dots, u_l\}$ for $U$, $\beta=\{v_1,\dots, v_m\}$ for $V$, and $\gamma=\{w_1,\dots, w_n\}$ for $W$ then the matrices of $A$ and $B$ with respect to these bases are

$[A]_\beta^\gamma = \begin{pmatrix} a_{11} & \cdots & a_{1m} \\ \vdots & & \vdots \\ a_{n1} & \cdots & a_{nm} \end{pmatrix}, \qquad [B]_\alpha^\beta = \begin{pmatrix} b_{11} & \cdots & b_{1l} \\ \vdots & & \vdots \\ b_{m1} & \cdots & b_{ml} \end{pmatrix}$

$Av_j = a_{1j}w_1+\cdots+a_{nj}w_n,\qquad Bu_k= b_{1k}v_1+\cdots+b_{mk}v_m$

To find the matrix of $AB$ with respect to the bases $\alpha, \gamma$ we express $AB(u_i)$ in terms of $\{w_1, \dots, w_n\}$:

$\begin{aligned} AB(u_k) =&\; A\bigl(Bu_k\bigr) \\ =&\; A\left(b_{1k}v_1+\cdots+b_{mk}v_m \right) \\ =&\; b_{1k}Av_1+\cdots+b_{mk}Av_m \\ =&\; b_{1k}\left\{a_{11}w_1+\cdots+a_{n1}w_n\right\}+\\ &\; b_{2k}\left\{a_{12}w_1+\cdots+a_{n2}w_n\right\}+\\ &\;\quad\vdots\quad+\\ &\; b_{mk}\left\{a_{1m}w_1+\cdots+a_{nm}w_n\right\} \\ =&\; \left\{a_{11}b_{1k} + \cdots + a_{1m}b_{mk}\right\} w_1 + \cdots + \left\{a_{n1}b_{1k} + \cdots + a_{nm}b_{mk}\right\} w_n \end{aligned}$

This shows that the $k^{\rm th}$ column of the matrix $[AB]_\alpha^\gamma$ is given by

$\begin{pmatrix} a_{11}b_{1k} + \cdots + a_{1m}b_{mk} \\ \vdots \\ a_{n1}b_{1k} + \cdots + a_{nm}b_{mk} \end{pmatrix}$

Definition. If $\mathcal A = (a_{ij})$ is an $n\times l$ matrix and $\mathcal B = (b_{jk})$ is an $l\times m$ matrix then the matrix product $\mathcal A\mathcal B$ is defined to be the $n\times m$ matrix $\mathcal C=(c_{ik})$ whose entries are given by

Theorem. $[AB]_\alpha^\gamma = [A]_\beta^\gamma\, [B]_\gamma^\alpha$

Example. Let $R(\theta):\R^2\to\R^2$ be rotation through an angle $\theta$. Then the matrix of $R(\theta)$ is given by

$\mathcal{R} (\theta) = \begin{pmatrix} \cos \theta & -\sin \theta \\ \sin\theta & \cos \theta \end{pmatrix}$

If we first rotate by $\theta$ and then by $\phi$ we achieve the same as rotating by $\theta+\phi$. This implies

$\cR(\theta+\phi) = \cR(\theta)\cR(\phi).$

I.e.

$\begin{aligned} &\begin{pmatrix} \cos (\theta+\phi) & -\sin (\theta+\phi) \\ \sin(\theta+\phi) & \cos (\theta+\phi) \end{pmatrix}\\ &\qquad= \begin{pmatrix} \cos \theta & -\sin \theta \\ \sin\theta & \cos \theta \end{pmatrix} \begin{pmatrix} \cos \phi & -\sin \phi \\ \sin\phi & \cos \phi \end{pmatrix}\\ &\qquad= \begin{pmatrix} \cos \theta \cos\phi - \sin\theta\sin\phi & -\sin \theta\cos\phi-\cos\theta\sin\phi \\ \sin\theta\cos\phi+\cos\theta\sin\phi & \cos \theta \cos\phi - \sin\theta\sin\phi \end{pmatrix} \end{aligned}$

Thus we recover the addition formulas for sine and cosine:

$\begin{aligned} \cos(\theta+\phi) &= \cos \theta \cos\phi - \sin\theta \sin\phi \\ \sin(\theta+\phi) &= \sin\theta\cos\phi+\cos\theta\sin\phi \end{aligned}$

The vector space $\mathcal{L}(V,W)$

The set of all linear transformations from one vector space $V$ to another $W$, is itself a vector space over the same field $\mathbb{F}$. Addition is defined by saying that for any two linear maps $T,S:V\to W$ one has

$(T+S)(x) \stackrel{\rm def}{=} T(x)+S(x), \text{ for all }x\in V,$

and for any linear map $T:V\to W$ and any number $a\in\mathbb{F}$ one has

$(aT)(x) \stackrel{\rm def}{=} a\bigl(T(x)\bigr) \text{ for all }x\in V.$

Notation. $\cL(V,W)=\bigl\{T \mid T:V\to W \text{ is a linear transformation}\bigr\}$

If $V=W$ then one writes $\mathcal{L}(V)$ instead of $\mathcal{L}(V, W)$.

Theorem. If $T,S:V\to W$ are linear and if $a\in\F$ then the maps $T+S:V\to W$ and $aT:V\to W$ are linear. The set $\mathcal{L}(V , W)$ of linear maps $T:V\to W$ is a vector space.

The case in which $V=W$ is special because if $T,S:V\to V$ then not only are $T+S$ and $aT$ linear transformations $V\to V$, but the compositions $ST$ and $TS$ also are linear transformations from $V$ to itself.

Inverses and other powers of a linear transformation

Definition. A linear transformation $T:V\to W$ is called invertible if $T$ is both injective and surjective.

Theorem. If $T:V\to W$ is linear and invertible, then $T^{-1}:W\to V$ is also linear.