# Martingales 1

Some preliminary definitions: Let ${(\Omega, \mathcal F, \mathbb P)}$ be a probability space. An increasing sequence ${\{\mathcal F_n\}}$ of sub-${\sigma}$-algebras of ${\mathcal F}$ is called a filtration. A stochastic process ${\{X_n : n=1,2,\ldots \}}$ said to be adapted to a filtration ${\{\mathcal F_n\}}$ if ${X_n}$ is ${\mathcal F_n}$-measurable for each ${n}$ – that is, ${X^{-1}(B)\in\mathcal F_n}$ for an arbitrary Borel set ${B}$. The natural filtration associated with a process is defined by ${\mathcal F_n=\sigma(X_1,\ldots, X_n)}$.

Now, the more interesting ones. A discrete-time process ${\{X_n\}}$ is said to be a martingale with respect to a filtration ${\{\mathcal F_n \}}$ if

• ${\{X_n \}}$ is adapted to ${\{F_n \}}$.
• ${X_n}$ is integrable for all ${n}$ (i.e. ${\mathbb E[|X_n|]<\infty}$).
• ${\mathbb E[X_{n+1}\mid \mathcal F_n] = \mathbb E[X_n]}$ for all ${n}$.

Recall that the conditional expectation of a random variable ${X}$ with respect to a ${\sigma}$-algebra ${\mathcal G}$, denoted ${\mathbb E[X\mid\mathcal G]}$, is any random variable that satisfies ${\mathbb E[\mathbb E[X\mid\mathcal G]\mathsf 1_G ] = \mathbb E[X\mathsf 1_G ]}$ for all ${G\in\mathcal G}$, that is,

$\displaystyle \int_G \mathbb E[X\mid\mathcal G ]\ \mathsf d\mathbb P = \int_G X\ \mathsf d\mathbb P.$

For example, if ${\{X_n\}}$ is an i.i.d. sequence with ${\mathbb E[X_1]=0}$ and ${S_n:=\sum_{k=1}^n X_k}$, then

$\displaystyle \mathbb E[S_{n+1}\mid \mathcal F_n]= \mathbb E[X_{n+1}\mathcal F_n] + \mathbb E[S_n\mid\mathcal F_n]= \mathbb E[X_{n+1}] + \mathbb E[S_n] = 0,$

so the unbiased 1-${D}$ random walk is a martingale. For a biased random walk on ${\mathbb Z}$ with ${\mathbb P(X_1=1)=p=1-\mathbb P(X_1=-1)}$, it is clear that ${\{S_n\}}$ is not a martingale. However, letting ${W_n = h(S_n)}$ where ${h(x)= \left(\frac{1-p}p \right)^x}$, we have

\displaystyle \begin{aligned} \mathbb E[W_1] &= \mathbb E[W_1\mid X_1=-1]\mathbb P(X_1=-1) +\mathbb E[W_1\mid X_1=1]\mathbb P(X_1=1)\\ &= \left(\frac p{1-p}\right)(1-p) + \left(\frac{1-p}p\right)p\\ &= p + (1-p) = 1 \end{aligned}

and

\displaystyle \begin{aligned} \mathbb E[W_{n+1}] &= \mathbb E\left[\left(\frac{1-p}p\right)^{S_{n+1}} \right] \\ &= \mathbb E\left[\left(\frac{1-p}p\right)^{S_{n}}\left(\frac{1-p}p\right)^{X_{n+1}} \right] \\ &= \mathbb E\left[W_n\left(\frac{1-p}p\right)^{X_{n+1}}\mid X_{n+1}=-1\right]\mathbb P(X_{n+1}=-1)\\ &\quad+ \mathbb E\left[W_n\left(\frac{1-p}p\right)^{X_{n+1}}\mid X_{n+1}=1\right]\mathbb P(X_{n+1}=1)\\ &= \mathbb E[W_n]\left(\left(\frac p{1-p}\right)(1-p) + \left(\frac{1-p}p\right)p \right)\\ &= \mathbb E[W_n], \end{aligned}

so that ${\{W_n\}}$ is integrable, and

\displaystyle \begin{aligned} \mathbb E[W_{n+1}\mid \mathcal F_n] &= \mathbb E\left[\left(\frac{1-p}p\right)^{S_{n}}\left(\frac{1-p}p\right)^{X_{n+1}}\mid F_n \right]\\ &= \mathbb E[W_n\mid \mathcal F_n]\mathbb E\left[\left(\frac{1-p}p\right)^{X_{n+1}} \right]\\ &= \mathbb E[W_n]. \end{aligned}

It follows that ${\{W_n\}}$ is a martingale. If ${A\in\mathcal F_{n-1}}$ and ${i,j\in\mathbb Z}$ then

\displaystyle \begin{aligned} \mathbb P(S_{n+1}=j\mid S_n=i, A) &= \mathbb P(S_{n+1}=j, S_n=j-1\mid S_n=i,A)\\ &\quad+\mathbb P(S_{n+1}=j,S_n=j+1\mid S_n=i,A)\\ &= \mathbb P( \mathbb P(X_{n+1}=1\mid )\mathsf 1_{\{j=i+1\}} + \mathbb P(X_{n+1}=-1)\mathsf 1_{\{j=i-1\}}\\ &= p\cdot\mathsf 1_{\{j=i+1\}} + (1-p)\cdot\mathsf 1_{\{j=i-1\}}\\ &= \mathbb P(S_{n+1}=j\mid S_n=i), \end{aligned}

so ${\{S_n\}}$ is a Markov chain. Further, for any ${i\in\mathbb Z}$,

\displaystyle \begin{aligned} \sum_{j\in\mathbb Z} P_{ij}h(j) &= p\left( \frac{1-p}p \right)^{i+1} + (1-p)\left( \frac{1-p}p \right)^{i-1}\\ &= \left( \frac{1-p}p\right)^{i-1} \left(\frac{(1-p)^2}{p}+(1-p) \right)\\ &= \left( \frac{1-p}p \right)^i\left(\frac{(1-p)^2+p(1-p)}{p} \right)\\ &= \left( \frac{1-p}p \right)^i\left( \frac{1-p}p \right)\\ &= \left( \frac{1-p}p \right)^{i+1}\\ &= h(i), \end{aligned}

so that ${h}$ is a harmonic function. In general, if ${h}$ is a harmonic function for a Markov chain ${\{X_n\}}$, then ${h(X_n)}$ is a martingale. This follows from the Markov property:

$\displaystyle \mathbb E[h(X_{n+1})\mid \mathcal F_n] = \mathbb E[f(X_{n+1})\mid X_n] = \sum_j P_{X_n,j}h(j) = h(X_n).$