Exponential Distribution

The exponential distribution with rate {\lambda>0} is defined by

\displaystyle  F(x) = (1- e^{-\lambda x}) 1_{\{x>0\}}.

Since {F} is differentiable, this is a continuous probability distribution, with density

\displaystyle  f(x) = \lambda e^{-\lambda x} 1_{\{x>0\}}.

One of the main reason this distribution is interesting is the so-called memoryless property:

Theorem 1 If {X\sim\mathrm{Exp}(\lambda)} and {s,t\geqslant 0}, then

\displaystyle \mathbb P(X>s+t|X>s) = \mathbb P(X>t).

Proof: We compute

\displaystyle \begin{aligned} \mathbb P(X>s+t|X>s) &= \frac{\mathbb P(X>s+t)}{\mathbb P(X>s)}\\ &= \frac{e^{-\lambda(s+t)}}{e^-{\lambda s}}\\ &= e^{-\lambda t}\\ &= \mathbb P(X>t). \end{aligned}

\Box

To compute the moments of the exponential distribution, we will make use of the following lemma:

Theorem 2 If {\mathbb P(X\geqslant 0)=1}, then for any integer {k\geqslant 0},

\displaystyle \mathbb E[X^{k+1}] = (k+1)\int_0^\infty t^k(1 - F(t))\mathsf dt,

where {F} is the distribution function of {X}.

Proof: By the Fubini-Tonelli theorem, we have

\displaystyle \begin{aligned} \int_0^\infty t^k (1-F(t))\mathsf dt &= \int_0^\infty t^k \mathbb E\left[1_{\{\omega\in\Omega: X(\omega)\geqslant t \}} \right]\mathsf dt\\ &= \int_0^\infty t^k\int_\Omega 1_{\{\omega\in\Omega: X(\omega)\geqslant t \}}\mathsf d\mathbb P\,\mathsf dt\\ &= \int_\Omega \int_0^{X(\omega)}t^k\mathsf dt\,\mathsf d\mathbb P\\ &= \int_\Omega \frac1{k+1} X^{k+1}(\omega)\mathsf d\mathbb P(\omega)\\ &= \frac1{k+1}\mathbb E[X^{k+1}], \end{aligned}

from which the result follows. \Box

We now need to compute

\displaystyle \int_0^\infty t^k e^{-\lambda t}\mathsf dt.

We show by induction that this is equal to {\frac{k!}{\lambda^{k+1}}}. For {k=0}, we have

\displaystyle \int_0^\infty t^0 e^{-\lambda t}\mathsf dt = \frac1\lambda = \frac{0!}{\lambda^1}.

Assume the claim is true for some nonnegative integer {k}, then

\displaystyle \begin{aligned} \int_0^\infty t^{k+1} e^{-\lambda t}\mathsf dt &= \left.-\frac{t^{k+1}}{\lambda}e^{-\lambda t}\right|_0^\infty + \frac{k+1}{\lambda}\int_0^\infty t^k e^{-\lambda}\mathsf dt\\ &= \left(\frac{k+1}\lambda \right)\left(\frac{k!}{\lambda^{k+1}} \right)\\ &= \frac{(k+1)!}{\lambda^{k+2}}. \end{aligned}

It follows then that

\displaystyle \mathbb E[X^{k+1}] = (k+1)\frac{k!}{\lambda^{k+1}} = \frac{(k+1)!}{\lambda^{k+1}}

for {k=0,1,2,\ldots}. To verify, let’s use another method. The moment-generating function of a random variable {X} is

\displaystyle M_X(t) := \mathbb E\left[e^{tX} \right],

for all {t} such that this expectation exists. Since

\displaystyle e^{tX}=\sum_{n=0}^\infty \frac{(tX)^n}{n!},

by nonnegativity and monotonicity of the above series, we may use the monotone convergence theorem to compute

\displaystyle \mathbb E\left[e^{tX} \right] = \mathbb E\left[\sum_{n=0}^\infty \frac{(tX)^n}{n!} \right] = \sum_{n=0}^\infty \frac{t^n \mathbb E[X^n] }{n!}.

Since this series converges absolutely, for any positive integer {k},

\displaystyle \begin{aligned} \frac{\mathsf d^k}{\mathsf dt^k}\left[M_X(t)\right] &= \frac{\mathsf d^k}{\mathsf dt^k}\left[\sum_{n=0}^\infty \frac{t^n \mathbb E[X^n] }{n!}\right]\\ &= \sum_{n=0}^\infty\frac{\mathsf d^k}{\mathsf dt^k} \left[\frac{t^n \mathbb E[X^n] }{n!}\right]\\ &= \sum_{n=0}^\infty \frac{\prod_{j=0}^{k-1}(n-j)}{n!}t^{n-k}\mathbb E[X^n]\\ &= \sum_{n=0}^\infty \frac{1}{(n-k)!}t^{n-k}\mathbb E[X^n]\\ \end{aligned}

Setting {t=0}, all terms except for {n=k} are zero, and hence

\displaystyle M_X^{(k)}(0) = \mathbb E[X^k].

The moment-generating function of the exponential distribution is

\displaystyle \begin{aligned} M_X(t) &= \int_0^\infty e^{tx}\lambda e^{-\lambda x}\mathsf dx\\ &= \int_0^\infty \lambda e^{-(\lambda-t)x}\mathsf dx\\ &= \frac{\lambda}{\lambda-t},\quad t<\lambda. \end{aligned}

Hence

\displaystyle M_X^{(k+1)}(t) = \frac{(k+1)!}{(\lambda - t)^{k+1}},

so that

\displaystyle \mathbb E[X^{k+1}] = M_X^{(k+1)}(0) = \frac{(k+1)!}{\lambda^{k+1}}.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s