-->
Math 519 Linear Systems of Differential Equations
First Order Homogeneous Linear Systems
A linear homogeneous system of differential equations is a system of
the form
\[
\begin{aligned}
\dot x_1 &= a_{11}x_1 + \cdots + a_{1n}x_n \\
\dot x_2 &= a_{21}x_1 + \cdots + a_{2n}x_n \\
&\;\;\vdots\\
\dot x_n &= a_{n1}x_1 + \cdots + a_{nn}x_n
\end{aligned}
\]
which we can abbreviate in matrix form to
\[
\dot\vx = A\vx,
\]
where
\[
\vx=
\begin{bmatrix}
x_1\\\vdots\\x_n
\end{bmatrix}
\qquad
A=
\begin{bmatrix}
a_{11} & \dots & a_{1n}\\
& \dots & \\
a_{n1} & \dots & a_{nn}
\end{bmatrix}
\]
Linear systems $\dot\vx = A\vx$ can always be solved, and their solution can be
written in the same way as the solution of the single differential equation
$\dot x=ax$, namely
\[
\vx(t) = e^{tA}\vx(0),
\]
provided we define the exponential $e^{tA}$ of the matrix $tA$ appropriately.
Below we first define $e^A$ for any $n\times n$
matrix $A$, and then give three different but equivalent ways of
presenting the solution to $\vx'(t)=A\vx(t)$.
The matrix exponential
The Series
There are several equivalent definitions of $e^{A}$ for a matrix $A$.
One of these is to define $e^{A}$ by substituting $A$ in the Taylor series for
$e^x$. Thus we define
\[
e^A = \exp (A) = I+A+\frac{1}{2!}A^2+\frac{1}{3!}A^3+\cdots
= \sum_{k=0}^\infty
\frac{A^k}{k!}\;.
\]
Since we can multiply the matrix $A$ with itself any number of times, each term
\[
A^k = \overbrace{A\cdot A\cdot \dots \cdot A}^{k\text{ times}}
\]
in the series is well defined. To show that the series converges it
is convenient to introduce “norms” of vectors and
matrices. For now we will compute with the series and ignore
questions about convergence.
Properties of the matrix exponential
Fact. If $O$ is the $n\times n$
zero-matrix, and $I$ is the $n\times n$ identity matrix, then $e^O =
I$
This follows by direct substitution of $A=O$ in the series for $e^A$.
Theorem.
$\displaystyle\frac{d}{dt}e^{tA} = Ae^{tA}$
Proof.
By definition we have
\[
e^{tA} = \sum_{n=0}^{\infty} t^n \frac{A^n}{n!}
\]
which implies
\[
\frac{d}{dt}e^{tA} = \frac{d}{dt}\Bigl(I+tA + t^2 \frac{A^2}{2!} + \cdots\Bigr)
=O +
A + t A^2 + t^2 \frac{A^3}{2!} + \cdots
=Ae^{tA}.
\]
For a complete proof we should also justify
Theorem.
If $AB=BA$ then $e^{A+B}=e^A e^B$, and also $e^A B = Be^A$.
Proof.
We substitute $A+B$ in the definition of the exponential and get
\begin{equation}
e^{A+B} = \sum_{n=0}^\infty \frac{(A+B)^n}{n!}.
\label{eqExpApB}
\end{equation}
The next step is to expand $(A+B)^n$ using the binomial theorem. This only
works when $A$ and $B$ commute, e.g.
\begin{align*}
(A+B)^2 &= (A+B)(A+B) \\
&= A^2 + AB + BA + B^2 \\
&= A^2 + AB + AB + B^2 \\
&= A^2 + 2AB + B^2.
\end{align*}
Note that we used $AB=BA$ here. In general one has
\[
(A+B)^n =
A^n + \binom n1 A^{n-1}B + \cdots + \binom n{n-1} A B^{n-1} + B^n.
\]
where
\[
\binom nk = \frac{n!}{k!(n-k)!}
\]
are the usual binomial
coefficients.
Applying this to (\ref{eqExpApB}) we get
\[
e^{A+B} = \sum_{n=0}^\infty \sum_{k=0}^n \frac{1}{n!} \frac{n!}{k!(n-k)!} A^k
B^{n-k}
=\sum_{n=0}^\infty \sum_{k=0}^n \frac{A^k}{k!} \frac{B^{n-k}}{(n-k)!}.
\]
Switch the order of summation:
\[
e^{A+B} = \sum_{k=0}^\infty \sum_{n=k}^{\infty} \frac{A^k}{k!} \frac{B^{n-k}}{(n-k)!}.
\]
and relabel the terms in the inner summation by setting $n=m+k$:
\[
e^{A+B} = \sum_{k=0}^\infty \sum_{m=0}^{\infty} \frac{A^k}{k!} \frac{B^{m}}{m!}
=\sum_{k=0}^\infty \frac{A^k}{k!} \,
\sum_{m=0}^\infty \frac{B^m}{m!}
= e^A e^B.
\]
q.e.d.
Theorem.
If $T$ is invertible, then $Te^A T^{-1} = e^{TAT^{-1}}$.
Solution in terms of the matrix exponential
The following theorem presents the solution of our linear homogeneous
differential equation
\begin{equation}
\frac{d\vx}{dt} = A\vx(t), \qquad \vx(0)=\vx_0.
\label{eq:linear-homogeneous}
\end{equation}
for any matrix $A$ and for any initial condition $\vx_0$. In practical
situations where you want to compute the solutions for some given matrix $A$
this solution is only useful if you have a good method of finding $e^{tA}$.
Theorem.
The unique solution of \eqref{eq:linear-homogeneous} is
\[
\vx(t) = e^{tA}\vx_0.
\]
Proof. Using the properties of the exponential $e^{tA}$ it
is easy to check that $\vx(t) = e^{tA}\vx_0$ satisfies $\vx'(t) =
A\vx(t)$, and also that $\vx(0)=\vx_0$, so $\vx(t) = e^{tA}\vx_0$
certainly is a solution.
To see that there are no other solutions,
assume that $\vx(t)$ is a solution and consider $\vy(t) =
e^{-tA}\vx(t)$. Then we have
\begin{align*}
\frac{d\vy(t)} {dt} &= \frac{d} {dt}\bigl(e^{-tA}\vx(t)\bigr)\\
&=Ae^{-tA}\vx(t) - e^{-tA}\vx'(t) \\
&= Ae^{-tA}\vx(t) - e^{-tA}A\vx(t)\\
&=0
\end{align*}
because $A$ commutes with $e^{tA}$. Thus if $\vx(t)$ is a solution,
then $\vy (t)$ is constant. At $t=0$ we have $\vy(0) = \vx(0)=\vx_0$,
and therefore $\vy(t)=\vx_0$ for all $t$. Since, by definition, $\vy(t) =
e^{-tA}\vx(t)$, we get $e^{-tA}\vx(t) = \vx_0$ for all $t$. Multiplying both
sides with $e^{tA}$ then leads to $\vx(t) = e^{tA}\vx_0$.
q.e.d.
Solution in terms of eigenvectors and eigenvalues
Another way to describe the solution of \eqref{eq:linear-homogeneous}
is to use the eigenvalues and eigenvectors of the matrix $A$.
The relevant facts from linear algebra about eigenvalues/vectors are
summarized here.
We make the following assumptions
about the matrix $A$:
- the matrix $A$ has $n$ eigenvectors $\vv_1$, … , $\vv_n$, with
corresponding eigenvalues $\lambda_1$, …, $\lambda_n$.
- the eigenvectors of $A$ form a basis of $\R^n$.
Computing the Exponential by Diagonalization
We continue to assume, as above, that $A$ is an $n\times n$ matrix for which we
have a basis of eigenvectors $\{\vv_1, \ldots, \vv_n\}$. If the eigenvectors,
when written in components, are
\[
\vv_1=
\begin{bmatrix}
v_{11}\\ \vdots \\ v_{n1}
\end{bmatrix},\quad
\vv_2=
\begin{bmatrix}
v_{12}\\ \vdots \\ v_{n2}
\end{bmatrix},\quad\dots\quad
\vv_n=
\begin{bmatrix}
v_{1n}\\ \vdots \\ v_{nn}
\end{bmatrix},
\]
then we define $V$ to be the matrix whose columns are the eigenvectors
$\vv_1$, …, $\vv_n$, i.e.,
\[
V=
\begin{bmatrix}
v_{11} & v_{12} & \cdots & v_{1n} \\
v_{21} & v_{22} & & v_{2n} \\
& & \ddots & \vdots \\
v_{n1} & v_{n2} & \cdots & v_{nn}
\end{bmatrix}
\]
Fact. $AV=VD$ where $D$ is the diagonal matrix
\[
D =
\begin{bmatrix}
\lambda_1 & 0 & \dots &0\\
0&\lambda_2 & \dots &0\\
& & \ddots &\\
0&\dots & 0 & \lambda_n
\end{bmatrix}\, .
\]
Since the matrix $V$ is invertible, we have
\[
A = VDV^{-1}.
\]
The theorem on $Te^AT^{-1}$ then implies:
Theorem. The matrix exponential of $tA$ is given by
\[
e^{tA} = V e^{tD} V^{-1}.
\]
This is a useful fact since the exponential $e^{tD}$ of a diagonal matrix is
particularly easy:
\[
e^{tD} =
\begin{bmatrix}
e^{\lambda_1 t} & 0 & \dots &0\\
0&e^{\lambda_2t} & \dots &0\\
& & \ddots &\\
0&\dots & 0 & e^{\lambda_n t}
\end{bmatrix}\, .
\]
Exponentials of block diagonal matrices
Consider, as an example, the matrix
\[
A =
\begin{bmatrix}
a & b & 0 & 0 \\
c & d & 0 &0 \\
0&0& p & q\\
0&0& r & s
\end{bmatrix}.
\]
This matrix has block diagonal form: it can be thought of as
\[
A =
\begin{bmatrix}
P & 0_{2\times2} \\ 0_{2\times2} & Q
\end{bmatrix}
\]
where the two zeros stand for the $2\times 2$ zero matrix, and where
\[
P =
\begin{bmatrix}
a & b \\ c & d
\end{bmatrix},\qquad
Q =
\begin{bmatrix}
p & q \\ r & s
\end{bmatrix}.
\]
One can check that the rule for multiplication of block matrices is the same as
for ordinary matrices, e.g. if $P,Q,R,S$ are $2\times2$ matrices, then
\[
\begin{bmatrix}
P & \\ & Q
\end{bmatrix}
\begin{bmatrix}
R & \\ & S
\end{bmatrix}
=
\begin{bmatrix}
PR & \\ & QS
\end{bmatrix}.
\]
In particular,
\[
\begin{bmatrix}
P & \\ & Q
\end{bmatrix}^n
=
\begin{bmatrix}
P^n & \\ & Q^n
\end{bmatrix},
\]
and thus
\[
\exp\left( t
\begin{bmatrix}
P & \\ & Q
\end{bmatrix}
\right)
=
\begin{bmatrix}
e^{tP} & \\ & e^{tQ}
\end{bmatrix}.
\]
Example
Compute the exponential $e^{tA}$ for the matrix
\[
A =
\begin{bmatrix}
-1 & 0 & 0 \\
0 & 0 & 1 \\
0 & -1 & 0
\end{bmatrix}.
\]
This matrix is block diagonal of the form
\[
A =
\begin{bmatrix}
P & \\ & Q
\end{bmatrix}
\]
where $P$ is the “$1\times1$ matrix” $P = [-1]$, and $Q$ is the
$2\times 2$ matrix
\[
Q =
\begin{bmatrix}
0 & 1 \\ -1 & 0
\end{bmatrix}.
\]
We know the exponentials of both $P$ and $Q$ (namely, $e^{tP} = [e^{-t}]$, and see the page on complex
eigenvalues for the computation of $e^{tJ}$.)
Therefore
\[
e^{tA}
=
\begin{bmatrix}
e^{tP} & \\ & e^{tQ}
\end{bmatrix}
=
\begin{bmatrix}
e^{-t} & 0 & 0\\
0 & \cos t & \sin t \\
0 & -\sin t & \cos t
\end{bmatrix}.
\]