Martingales 1

Some preliminary definitions: Let {(\Omega, \mathcal F, \mathbb P)} be a probability space. An increasing sequence {\{\mathcal F_n\}} of sub-{\sigma}-algebras of {\mathcal F} is called a filtration. A stochastic process {\{X_n : n=1,2,\ldots \}} said to be adapted to a filtration {\{\mathcal F_n\}} if {X_n} is {\mathcal F_n}-measurable for each {n} – that is, {X^{-1}(B)\in\mathcal F_n} for an arbitrary Borel set {B}. The natural filtration associated with a process is defined by {\mathcal F_n=\sigma(X_1,\ldots, X_n)}.

Now, the more interesting ones. A discrete-time process {\{X_n\}} is said to be a martingale with respect to a filtration {\{\mathcal F_n \}} if

  • {\{X_n \}} is adapted to {\{F_n \}}.
  • {X_n} is integrable for all {n} (i.e. {\mathbb E[|X_n|]<\infty}).
  • {\mathbb E[X_{n+1}\mid \mathcal F_n] = \mathbb E[X_n]} for all {n}.

Recall that the conditional expectation of a random variable {X} with respect to a {\sigma}-algebra {\mathcal G}, denoted {\mathbb E[X\mid\mathcal G]}, is any random variable that satisfies {\mathbb E[\mathbb E[X\mid\mathcal G]\mathsf 1_G ] = \mathbb E[X\mathsf 1_G ]} for all {G\in\mathcal G}, that is,

\displaystyle \int_G \mathbb E[X\mid\mathcal G ]\ \mathsf d\mathbb P = \int_G X\ \mathsf d\mathbb P.

For example, if {\{X_n\}} is an i.i.d. sequence with {\mathbb E[X_1]=0} and {S_n:=\sum_{k=1}^n X_k}, then

\displaystyle \mathbb E[S_{n+1}\mid \mathcal F_n]= \mathbb E[X_{n+1}\mathcal F_n] + \mathbb E[S_n\mid\mathcal F_n]= \mathbb E[X_{n+1}] + \mathbb E[S_n] = 0,

so the unbiased 1-{D} random walk is a martingale. For a biased random walk on {\mathbb Z} with {\mathbb P(X_1=1)=p=1-\mathbb P(X_1=-1)}, it is clear that {\{S_n\}} is not a martingale. However, letting {W_n = h(S_n)} where {h(x)= \left(\frac{1-p}p \right)^x}, we have

\displaystyle  \begin{aligned} \mathbb E[W_1] &= \mathbb E[W_1\mid X_1=-1]\mathbb P(X_1=-1) +\mathbb E[W_1\mid X_1=1]\mathbb P(X_1=1)\\ &= \left(\frac p{1-p}\right)(1-p) + \left(\frac{1-p}p\right)p\\ &= p + (1-p) = 1 \end{aligned}

and

\displaystyle  \begin{aligned} \mathbb E[W_{n+1}] &= \mathbb E\left[\left(\frac{1-p}p\right)^{S_{n+1}} \right] \\ &= \mathbb E\left[\left(\frac{1-p}p\right)^{S_{n}}\left(\frac{1-p}p\right)^{X_{n+1}} \right] \\ &= \mathbb E\left[W_n\left(\frac{1-p}p\right)^{X_{n+1}}\mid X_{n+1}=-1\right]\mathbb P(X_{n+1}=-1)\\ &\quad+ \mathbb E\left[W_n\left(\frac{1-p}p\right)^{X_{n+1}}\mid X_{n+1}=1\right]\mathbb P(X_{n+1}=1)\\ &= \mathbb E[W_n]\left(\left(\frac p{1-p}\right)(1-p) + \left(\frac{1-p}p\right)p \right)\\ &= \mathbb E[W_n], \end{aligned}

so that {\{W_n\}} is integrable, and

\displaystyle  \begin{aligned} \mathbb E[W_{n+1}\mid \mathcal F_n] &= \mathbb E\left[\left(\frac{1-p}p\right)^{S_{n}}\left(\frac{1-p}p\right)^{X_{n+1}}\mid F_n \right]\\ &= \mathbb E[W_n\mid \mathcal F_n]\mathbb E\left[\left(\frac{1-p}p\right)^{X_{n+1}} \right]\\ &= \mathbb E[W_n]. \end{aligned}

It follows that {\{W_n\}} is a martingale. If {A\in\mathcal F_{n-1}} and {i,j\in\mathbb Z} then

\displaystyle \begin{aligned} \mathbb P(S_{n+1}=j\mid S_n=i, A) &= \mathbb P(S_{n+1}=j, S_n=j-1\mid S_n=i,A)\\ &\quad+\mathbb P(S_{n+1}=j,S_n=j+1\mid S_n=i,A)\\ &= \mathbb P( \mathbb P(X_{n+1}=1\mid )\mathsf 1_{\{j=i+1\}} + \mathbb P(X_{n+1}=-1)\mathsf 1_{\{j=i-1\}}\\ &= p\cdot\mathsf 1_{\{j=i+1\}} + (1-p)\cdot\mathsf 1_{\{j=i-1\}}\\ &= \mathbb P(S_{n+1}=j\mid S_n=i), \end{aligned}

so {\{S_n\}} is a Markov chain. Further, for any {i\in\mathbb Z},

\displaystyle \begin{aligned} \sum_{j\in\mathbb Z} P_{ij}h(j) &= p\left( \frac{1-p}p \right)^{i+1} + (1-p)\left( \frac{1-p}p \right)^{i-1}\\ &= \left( \frac{1-p}p\right)^{i-1} \left(\frac{(1-p)^2}{p}+(1-p) \right)\\ &= \left( \frac{1-p}p \right)^i\left(\frac{(1-p)^2+p(1-p)}{p} \right)\\ &= \left( \frac{1-p}p \right)^i\left( \frac{1-p}p \right)\\ &= \left( \frac{1-p}p \right)^{i+1}\\ &= h(i), \end{aligned}

so that {h} is a harmonic function. In general, if {h} is a harmonic function for a Markov chain {\{X_n\}}, then {h(X_n)} is a martingale. This follows from the Markov property:

\displaystyle \mathbb E[h(X_{n+1})\mid \mathcal F_n] = \mathbb E[f(X_{n+1})\mid X_n] = \sum_j P_{X_n,j}h(j) = h(X_n).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s