# Exponential Distribution

The exponential distribution with rate ${\lambda>0}$ is defined by

$\displaystyle F(x) = (1- e^{-\lambda x}) 1_{\{x>0\}}.$

Since ${F}$ is differentiable, this is a continuous probability distribution, with density

$\displaystyle f(x) = \lambda e^{-\lambda x} 1_{\{x>0\}}.$

One of the main reason this distribution is interesting is the so-called memoryless property:

Theorem 1 If ${X\sim\mathrm{Exp}(\lambda)}$ and ${s,t\geqslant 0}$, then

$\displaystyle \mathbb P(X>s+t|X>s) = \mathbb P(X>t).$

Proof: We compute

\displaystyle \begin{aligned} \mathbb P(X>s+t|X>s) &= \frac{\mathbb P(X>s+t)}{\mathbb P(X>s)}\\ &= \frac{e^{-\lambda(s+t)}}{e^-{\lambda s}}\\ &= e^{-\lambda t}\\ &= \mathbb P(X>t). \end{aligned}

$\Box$

To compute the moments of the exponential distribution, we will make use of the following lemma:

Theorem 2 If ${\mathbb P(X\geqslant 0)=1}$, then for any integer ${k\geqslant 0}$,

$\displaystyle \mathbb E[X^{k+1}] = (k+1)\int_0^\infty t^k(1 - F(t))\mathsf dt,$

where ${F}$ is the distribution function of ${X}$.

Proof: By the Fubini-Tonelli theorem, we have

\displaystyle \begin{aligned} \int_0^\infty t^k (1-F(t))\mathsf dt &= \int_0^\infty t^k \mathbb E\left[1_{\{\omega\in\Omega: X(\omega)\geqslant t \}} \right]\mathsf dt\\ &= \int_0^\infty t^k\int_\Omega 1_{\{\omega\in\Omega: X(\omega)\geqslant t \}}\mathsf d\mathbb P\,\mathsf dt\\ &= \int_\Omega \int_0^{X(\omega)}t^k\mathsf dt\,\mathsf d\mathbb P\\ &= \int_\Omega \frac1{k+1} X^{k+1}(\omega)\mathsf d\mathbb P(\omega)\\ &= \frac1{k+1}\mathbb E[X^{k+1}], \end{aligned}

from which the result follows. $\Box$

We now need to compute

$\displaystyle \int_0^\infty t^k e^{-\lambda t}\mathsf dt.$

We show by induction that this is equal to ${\frac{k!}{\lambda^{k+1}}}$. For ${k=0}$, we have

$\displaystyle \int_0^\infty t^0 e^{-\lambda t}\mathsf dt = \frac1\lambda = \frac{0!}{\lambda^1}.$

Assume the claim is true for some nonnegative integer ${k}$, then

\displaystyle \begin{aligned} \int_0^\infty t^{k+1} e^{-\lambda t}\mathsf dt &= \left.-\frac{t^{k+1}}{\lambda}e^{-\lambda t}\right|_0^\infty + \frac{k+1}{\lambda}\int_0^\infty t^k e^{-\lambda}\mathsf dt\\ &= \left(\frac{k+1}\lambda \right)\left(\frac{k!}{\lambda^{k+1}} \right)\\ &= \frac{(k+1)!}{\lambda^{k+2}}. \end{aligned}

It follows then that

$\displaystyle \mathbb E[X^{k+1}] = (k+1)\frac{k!}{\lambda^{k+1}} = \frac{(k+1)!}{\lambda^{k+1}}$

for ${k=0,1,2,\ldots}$. To verify, let’s use another method. The moment-generating function of a random variable ${X}$ is

$\displaystyle M_X(t) := \mathbb E\left[e^{tX} \right],$

for all ${t}$ such that this expectation exists. Since

$\displaystyle e^{tX}=\sum_{n=0}^\infty \frac{(tX)^n}{n!},$

by nonnegativity and monotonicity of the above series, we may use the monotone convergence theorem to compute

$\displaystyle \mathbb E\left[e^{tX} \right] = \mathbb E\left[\sum_{n=0}^\infty \frac{(tX)^n}{n!} \right] = \sum_{n=0}^\infty \frac{t^n \mathbb E[X^n] }{n!}.$

Since this series converges absolutely, for any positive integer ${k}$,

\displaystyle \begin{aligned} \frac{\mathsf d^k}{\mathsf dt^k}\left[M_X(t)\right] &= \frac{\mathsf d^k}{\mathsf dt^k}\left[\sum_{n=0}^\infty \frac{t^n \mathbb E[X^n] }{n!}\right]\\ &= \sum_{n=0}^\infty\frac{\mathsf d^k}{\mathsf dt^k} \left[\frac{t^n \mathbb E[X^n] }{n!}\right]\\ &= \sum_{n=0}^\infty \frac{\prod_{j=0}^{k-1}(n-j)}{n!}t^{n-k}\mathbb E[X^n]\\ &= \sum_{n=0}^\infty \frac{1}{(n-k)!}t^{n-k}\mathbb E[X^n]\\ \end{aligned}

Setting ${t=0}$, all terms except for ${n=k}$ are zero, and hence

$\displaystyle M_X^{(k)}(0) = \mathbb E[X^k].$

The moment-generating function of the exponential distribution is

\displaystyle \begin{aligned} M_X(t) &= \int_0^\infty e^{tx}\lambda e^{-\lambda x}\mathsf dx\\ &= \int_0^\infty \lambda e^{-(\lambda-t)x}\mathsf dx\\ &= \frac{\lambda}{\lambda-t},\quad t<\lambda. \end{aligned}

Hence

$\displaystyle M_X^{(k+1)}(t) = \frac{(k+1)!}{(\lambda - t)^{k+1}},$

so that

$\displaystyle \mathbb E[X^{k+1}] = M_X^{(k+1)}(0) = \frac{(k+1)!}{\lambda^{k+1}}.$