Integrating and differentiating sequences and series

Fundamental Theorem of Calculus.

If $f:[a, b]\to\mathbb{R}$ is continuous, then

$$F(x) {\stackrel{\textrm{def}}=}\int_a^x f(\xi)\; d\xi$$

is a differentiable function, and

$$F'(x) = \frac{d}{dx}\int_a^x f(\xi)\; d\xi = f(x).$$

Switching and differentiating integrals

Switching integrals. If $f:[a,b]\times[c,d]\to\mathbb{R}$ is a Riemann integrable function of two variables, then

$$\int_a^b\int_c^d f(x, y)\;dy\;dx = \int_c^d\int_a^b f(x, y)\;dx\;dy$$

Integrals depending on a parameter. If $f : [a, b]\times(t_0, t_1)\to\mathbb{R}$ is continuous, then the function $F:(t_0, t_1)\to\mathbb{R}$ given by

$$F(t) {\stackrel{\textrm{def}}=}\int_a^b f(x, t)\;dx$$

is continuous.

Differentiating the parameter in an integral. If $f, \frac{\partial f}{\partial t} : [a, b]\times(t_0, t_1)\to\mathbb{R}$

are continuous, then

$$\frac{d}{dt}\int_a^b f( x, t)\; dx = \int_a^b \frac{\partial f}{\partial t}(x, t)\; dx$$

Proof. Let

$$F(t) = \int _a^b f(x, t)\;dx.$$

Choose $T\in (t_0, t_1)$. Then

$$\begin{aligned} F(t)& = F(T) + \int_a^b \bigl\{ f(x, t) - f(x, T)\bigr\}\; dx\\ &= F(T) + \int_a^b \int_T^t \frac{\partial f}{\partial t}(x, \tau)\;d\tau\;dx. \end{aligned}$$

Since $\partial f/\partial t$ is continuous we are allowed to switch the order of integration:

$$F(t) = F(T) + \int_T^t \int_a^b\frac{\partial f}{\partial t}(x, \tau)\;dx\;d\tau.$$

The Fundamental Theorem of Calculus now implies

$$F'(t) = \frac{d}{dt} \int_T^t \int_a^b\frac{\partial f}{\partial t}(x, \tau)\;dx\;d\tau =\int_a^b\frac{\partial f}{\partial t}(x, t)\;dx.$$

Q.E.D.

Norms and inner products

Definition (real inner product) A real inner product on a real vector space $V$ is a real valued function

on $V\times V$, usually written as $(x,y)$ or $\langle x, y\rangle$ such that the following properties hold for all $x,y,z\in V$ and $a\in\mathbb R$:

• $\langle x, y\rangle = \langle y, x\rangle$
• $\langle ax, y\rangle = a \langle x, y\rangle$
• $\langle x+y, z\rangle = \langle x, z\rangle + \langle y, z\rangle$
• $\langle x,x\rangle \gt 0$ if $x\neq 0$

Definition (complex inner product) A complex inner product on a complex vector space $V$ is a complex valued

function on $V\times V$, usually written as $(x,y)$ or $\langle x, y\rangle$ such that the following properties hold for all $x,y,z\in V$ and $a\in\mathbb C$:

• $\langle x, y\rangle = \overline{\langle y, x\rangle}$ where $\overline{z}$ is the complex conjugate of $z\in\mathbb C$
• $\langle ax, y\rangle = a \langle x, y\rangle$
• $\langle x+y, z\rangle = \langle x, z\rangle + \langle y, z\rangle$
• $\langle x,x\rangle \gt 0$ if $x\neq 0$

Example: inner product on a function space.

Let $H$ be the space of $2\pi$-periodic continuous functions $f:\R\to\mathbb C$. Then

$$\left\langle f,g\right\rangle \stackrel{\rm def}{=} \frac{1}{2\pi}\int_0^{2\pi} f(t)\overline{g(t)} \, dt$$

defines an inner product on $H$.

Definition (norm)

If $V$ is a real or complex vector space then a function $x\in V\mapsto \Vert x\Vert\in\R$ is called a norm on $V$ if

• $\|x\|\gt 0$ for all $x\in V$ with $x\neq 0$
• $\|ax\| = |a|\,\|x\|$ for all $x\in V$ and $a\in\R$ or $a\in\mathbb{C}$
• $\|x+y\|\leq \|x\|+\|y\|$     (the triangle inequality)

Definition (distance between vectors) The distance between two vectors $x, y\in V$ in a normed vector space is $d(x, y) \stackrel{\rm def}{=} \|x-y\|$.

Theorem about the norm in an inner product space

If $V$ is a real or complex inner product space with inner product $\left\langle x, y\right\rangle$. Then

$$\|x\| = \sqrt{\left\langle x, x\right\rangle} .$$

is a norm on $V$. It satisfies the Cauchy-Schwarz inequality

$$\left|\left\langle x, y \right\rangle\right|\leq \|x\|\,\|y\|$$

for all $x, y\in V$.

Note that $\langle x, x\rangle$ is never negative, so the square root is always defined.

Definition (orthogonality)

Two vectors $x,y\in V$ are said to be orthogonal if $\langle x, y\rangle =0$. Notation: $x\perp y$ means $x, y$ are orthogonal.

In particular, the zero vector is orthogonal to every other vector, because $\langle x, 0\rangle =0$ for all $x\in V$.

Convergence of sequences and series of functions

Uniform convergence

Let $V$ be the vector space of bounded functions from some set $X$ to the real numbers:

$$V = \bigl\{ f:X\to\R \mid \sup_{x\in X} |f(x)| <\infty \bigr\}.$$

Then

$$\|f\|_\infty \stackrel{\sf def}{=} \sup_{x\in X}|f(x)|$$

defines a norm on $V$, often called the “sup–norm”. By definition a sequence of functions $f_n:X\to\R$ converges uniformly to a function $f:X\to\R$ if $\lim_{n\to\infty} \|f_n-f\|_\infty = 0$.

Weierstrass M-test

Theorem. Let $f_n:[a,b]\to\mathbb{C}$ be a sequence of functions for which we can find real numbers $M_n>0$ such that

• $|f_n(x)|\leq M_n$ for all $x\in [a,b]$, and
• $\sum_{n=1}^\infty M_n < \infty$.

Then the series $\sum_{n=1}^\infty f_n(x)$ converges uniformly on $[a,b]$.

Proof. Let $s(x)=\sum_{n=1}^\infty f_n(x)$ and $s_N(x)=\sum_{n=1}^Nf_n(x)$. We have to show that $s_N(x)$ converges uniformly to $s(x)$.

Let $\epsilon\gt 0$ be given. For any $x\in [a,b]$ and any $N\in\N$ we have

$$\begin{aligned} |s(x)-s_N(x)| &= \left\vert s(x) - \sum_{n=1}^N f_n(x) \right\vert \\ &= \left\vert \sum_{n=1}^\infty f_n(x) - \sum_{n=1}^N f_n(x) \right\vert \\ &= \left\vert \sum_{n=N+1}^\infty f_n(x) \right\vert \\ &\leq \sum_{n=N+1}^\infty |f_n(x)| \\ &\leq \sum_{n=N+1}^\infty M_n \end{aligned}$$

Since the series $\sum M_n$ is known to converge, an $N_\epsilon\in\N$ exists for which $\sum_{n=N_\epsilon}^\infty M_n <\epsilon$. It then follows that for all $N\geq N_\epsilon$ and all $x\in [a,b]$ one has

$$|s(x) - s_N(x)|< \epsilon.$$

This implies $s_N(x)\to s(x)$ uniformly.

Example 1. The series $\sum_{n=1}^\infty \dfrac{\sin nx}{n^2}$ converges uniformly for all $x\in\R$ because the $n^{\rm th}$ term is bounded by $\left\vert\dfrac{\sin nx}{n^2}\right\vert \leq M_n$ with $M_n = \frac1{n^2}$, and the series $\sum_1^\infty \frac{1}{n^2}$ converges.
Each term is continuous and the series converges uniformly. Therefore the function $f(x)=\sum_1^\infty \dfrac{\sin nx}{n^2}$ is a continuous function.

Example 2. The sawtooth function is defined by $f(x)=\frac{\pi-x}{2}$ ($0\lt x\lt 2\pi$) and $f(x+2\pi)=f(x)$ for all $x\in\R$. Its Fourier series is

$$f(x) = \sum_{n=1}^\infty \frac{\sin nx}{n}.$$

The best upper bound we can find for the $n^{\rm th}$ term is $M_n=\frac 1n$. The series $\sum_nM_n = \sum_n \frac1n$ does not converge, so we cannot apply the Weierstrass M-test.
The sawtooth function $f$ is discontinuous at $x=2k\pi$ ($k\in\Z$), so the series cannot converge uniformly.

The integral and derivative of a limit