Basic Theory

Basic Assumptions

For our basic ingredients, we start with a stochastic process $\mathit{X}=\{X_t:t\in T\}$ on an underlying probability space $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$, having state space $\mathbb{R}$, and where the index set $T$ (representing time) is either $\mathbb{N}$ (discrete time) or $[0,\mathrm{\infty })$ (continuous time). So to review what all this means, $\mathrm{\Omega }$ is the sample space, $\mathcal{F}$ the $\sigma$-algebra of events, $\mathbb{P}$ the probability measure on $(\mathrm{\Omega },\mathcal{F})$, and $X_t$ is a random variable with values in $\mathbb{R}$ for each $t\in T$. Next, we have a filtration $\mathfrak{F}=\{{\mathcal{F}}_t:t\in T\}$, and we assume that $\mathit{X}$ is adapted to $\mathfrak{F}$. To review again, $\mathfrak{F}$ is an increasing family of sub $\sigma$-algebras of $\mathcal{F}$, so that ${\mathcal{F}}_s\subseteq {\mathcal{F}}_t\subseteq \mathcal{F}$ for $s,t\in T$ with $s\le t$, and $X_t$ is measurable with respect to ${\mathcal{F}}_t$ for $t\in T$. We think of ${\mathcal{F}}_t$ as the collection of events up to time $t\in T$, thus encoding the information available at time $t$. Finally, we assume that $\mathbb{E}\left(\left|X_t\right|\right)<\mathrm{\infty }$, so that the mean of $X_t$ exists as a real number, for each $t\in T$.

There are two important special cases of the basic setup. The simplest case, of course, is when ${\mathcal{F}}_t=\sigma \{X_s:s\in T,s\le t\}$ for $t\in T$, so that $\mathfrak{F}$ is the natural filtration associated with $\mathit{X}$. Another case that arises frequently is when we have a second stochastic process $\mathit{Y}=\{Y_t:t\in T\}$ on $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ with values in a general measure space $(S,\mathcal{S})$, and $\mathfrak{F}$ is the natural filtration associated with $\mathit{Y}$. So in this case, our main assumption is that $X_t$ is measurable with respect to $\sigma \{Y_s:s\in T,s\le t\}$ for $t\in T$.

The theory of martingales is beautiful, elegant, and mostly accessible in discrete time, when $T=\mathbb{N}$. But as with the theory of Markov processes, martingale theory is technically much more complicated in continuous time, when $T=[0,\mathrm{\infty })$. In this case, additional assumptions about the continuity of the sample paths $t\mapsto X_t$ and the filtration $t\mapsto {\mathcal{F}}_t$ are often necessary in order to have a nice theory. Specifically, we will assume that the process$\mathit{X}$ is right continuous and has left limits, and that the filtration $\mathfrak{F}$ is right continuous and complete. These are the standard assumptions in continuous time.

Definitions

For the basic definitions that follow, you may need to review conditional expected value with respect to a $\sigma$-algebra.

In the special case that $\mathfrak{F}$ is the natural filtration associated with $\mathit{X}$, we simply say that $\mathit{X}$ is a martingale, without reference to the filtration. In the special case that we have a second stochastic process $\mathit{Y}=\{Y_t:t\in T\}$ and $\mathfrak{F}$ is the natural filtration associated with $\mathit{Y}$, we say that $\mathit{X}$ is a martingale with respect to $\mathit{Y}$.

The term martingale originally referred to a portion of the harness of a horse, and was later used to describe gambling strategies, such as the one used in the Petersburg paradox, in which bets are doubled when a game is lost. To interpret the definitions above in terms of gambling, suppose that a gambler is at a casino, and that $X_t$ represents her fortune at time $t\in T$ and ${\mathcal{F}}_t$ the information available to her at time $t$. Suppose now that $s,t\in T$ with $s<t$ and that we think of $s$ as the current time, so that $t$ is a future time. If $\mathit{X}$ is a martingale with respect to $\mathfrak{F}$ then the games are fair in the sense that the gambler's expected fortune at the future time $t$ is the same as her current fortune at time $s$. To venture a bit from the casino, suppose that $X_t$ is the price of a stock, or the value of a stock index, at time $t\in T$. If $\mathit{X}$ is a martingale, then the expected value at a future time, given all of our information, is the present value.

But as we will see, martingales are useful in probability far beyond the application to gambling and even far beyond financial applications generally. Indeed, martingales are of fundamental importance in modern probability theory. Here are two related definitions, with equality in the martingale condition replaced by inequalities.

In the gambling setting, a sub-martingale models games that are favorable to the gambler on average, while a super-martingale models games that are unfavorable to the gambler on average. To venture again from the casino, suppose that $X_t$ is the price of a stock, or the value of a stock index, at time $t\in T$. If $\mathit{X}$ is a sub-martingale, the expected value at a future time, given all of our information, is greater than the present value, and if $\mathit{X}$ is a super-martingale then the expected value at the future time is less than the present value. One hopes that a stock index is a sub-martingale.

Clearly $\mathit{X}$ is a martingale with respect to $\mathfrak{F}$ if and only if it is both a sub-martingale and a super-martingale. Finally, recall that the conditional expected value of a random variable with respect to a $\sigma$-algebra is itself a random variable, and so the equations and inequalities in the definitions should be interpreted as holding with probability 1. In this section generally, statements involving random variables are assumed to hold with probability 1.

The conditions that define martingale, sub-martingale, and super-martingale make sense if the index set $T$ is any totally ordered set. In some applications that we will consider later, $T=\{0,1,\dots ,n\}$ for fixed $n\in {\mathbb{N}}_{+}$. In the section on backwards martingales, $T=\{-n:n\in \mathbb{N}\}$ or $T=(-\mathrm{\infty },0]$. In the case of discrete time when $T=\mathbb{N}$, we can simplify the definitions slightly.

The relations that define martingales, sub-martingales, and super-martingales hold for the ordinary (unconditional) expected values.

So if $\mathit{X}$ is a martingale then $\mathit{X}$ has constant expected value, and this value is referred to as the mean of $\mathit{X}$.

Examples

The goal for the remainder of this section is to give some classical examples of martingales, and by doing so, to show the wide variety of applications in which martingales occur. We will return to many of these examples in subsequent sections. Without further ado, we assume that all random variables are real-valued, unless otherwise specified, and that all expected values mentioned below exist in $\mathbb{R}$. Be sure to try the proofs yourself before expanding the details.

Constant Sequence

Our first example is rather trivial, but still worth noting.

Partial Sums

For our next discussion, we start with one of the most basic martingales in discrete time, and the one with the simplest interpretation in terms of gambling. Suppose that $\mathit{V}=\{V_n:n\in \mathbb{N}\}$ is a sequence of independent random variables with $\mathbb{E}(|V_k|)<\mathrm{\infty }$ for $k\in \mathbb{N}$. Let $$X_n=\sum _{k=0}^n{V}_k,n\in \mathbb{N}$$

In terms of gambling, if $X_0=V_0$ is the gambler's initial fortune and $V_i$ is the gambler's net winnings on the $i$th game, then $X_n$ is the gamblers net fortune after $n$ games for $n\in {\mathbb{N}}_{+}$. But partial sum processes associated with independent sequences are important far beyond gambling. In fact, much of classical probability deals with partial sums of independent and identically distributed variables. The entire chapter on Random Samples explores this setting.

Note that $\mathbb{E}(X_n)=\sum _{k=0}^n\mathbb{E}(V_k)$. Hence condition (a) is equivalent to $n\mapsto \mathbb{E}(X_n)$ increasing, condition (b) is equivalent to $n\mapsto \mathbb{E}(X_n)$ decreasing, and condition (c) is equivalent to $n\mapsto \mathbb{E}(X_n)$ constant. Here is another martingale associated with the partial sum process, known as the second moment martingale.

So under the assumptions in this theorem, both $\mathit{X}$ and $\mathit{Y}$ are martingales. We will generalize the results for partial sum processes below in the discussion on processes with independent increments in .

Martingale Difference Sequences

In the last discussion, we saw that the partial sum process associated with a sequence of independent, mean 0 variables is a martingale. Conversely, every martingale in discrete time can be written as a partial sum process of uncorrelated mean 0 variables. This representation gives some significant insight into the theory of martingales generally. Suppose that $\mathit{X}=\{X_n:n\in \mathbb{N}\}$ is a martingale with respect to the filtration $\mathfrak{F}=\{{\mathcal{F}}_n:n\in \mathbb{N}\}$.

As promised, the martingale difference variables have mean 0, and in fact satisfy a stronger property.

Also as promised, if the martingale variables have finite variance, then the martingale difference variables are uncorrelated.

We now know that a discrete-time martingale is the partial sum process associated with a sequence of uncorrelated variables. Hence we might hope that there are martingale versions of the fundamental theorems that hold for a partial sum process associated with an independent sequence. This turns out to be true, and is a basic reason for the importance of martingales.

Discrete-Time Random Walks

Suppose that $\mathit{V}=\{V_n:n\in \mathbb{N}\}$ is a sequence of independent random variables with $\{V_n:n\in {\mathbb{N}}_{+}\}$ identically distributed. We assume that $\mathbb{E}(|V_n|)<\mathrm{\infty }$ for $n\in \mathbb{N}$ and we let $a$ denote the common mean of $\{V_n:n\in {\mathbb{N}}_{+}\}$. Let $\mathit{X}=\{X_n:n\in \mathbb{N}\}$ be the partial sum process associated with $\mathit{V}$ so that $$X_n=\sum _{i=0}^n{V}_i,n\in \mathbb{N}$$ This setting is a special case of the more general partial sum process considered in . The process $\mathit{X}$ is sometimes called a (discrete-time) random walk. The initial position $X_0=V_0$ of the walker can have an arbitrary distribution, but then the steps that the walker takes are independent and identically distributed. In terms of gambling, $X_0=V_0$ is the initial fortune of the gambler playing a sequence of independent and identical games. If $V_i$ is the amount won (or lost) on game $i\in {\mathbb{N}}_{+}$, then $X_n$ is the gambler's net fortune after $n$ games.

For the second moment martingale, suppose that $V_n$ has common mean $a=0$ and common variance $b^2<\mathrm{\infty }$ for $n\in {\mathbb{N}}_{+}$, and that $\text{var}(V_0)<\mathrm{\infty }$.

We will generalize the results for discrete-time random walks below, in the discussion on processes with stationary, independent increments.

Partial Products

Our next discussion is similar to the one on partial sum processes in , but with products instead of sums. So suppose that $\mathit{V}=\{V_n:n\in \mathbb{N}\}$ is an independent sequence of nonnegative random variables with $\mathbb{E}(V_n)<\mathrm{\infty }$ for $n\in \mathbb{N}$. Let $$X_n=\prod _{i=0}^n{V}_i,n\in \mathbb{N}$$ so that $\mathit{X}=\{X_n:n\in \mathbb{N}\}$ is the partial product process associated with $\mathit{X}$.

As with random walks, a special case of interest is when $\{V_n:n\in {\mathbb{N}}_{+}\}$ is an identically distributed sequence.

The Simple Random Walk

Suppose now that that $\mathit{V}=\{V_n:n\in \mathbb{N}\}$ is a sequence of independent random variables with $\mathbb{P}(V_i=1)=p$ and $\mathbb{P}(V_i=-1)=1-p$ for $i\in {\mathbb{N}}_{+}$, where $p\in (0,1)$. Let $\mathit{X}=\{X_n:n\in \mathbb{N}\}$ be the partial sum process associated with $\mathit{V}$ so that $$X_n=\sum _{i=0}^n{V}_i,n\in \mathbb{N}$$ Then $\mathit{X}$ is the simple random walk with parameter $p$, and of course, is a special case of the more general random walk studied in . In terms of gambling, our gambler plays a sequence of independent and identical games, and on each game, wins €1 with probability $p$ and loses €1 with probability $1-p$. So if $V_0$ is the gambler's initial fortune, then $X_n$ is her net fortune after $n$ games.

So case (a) corresponds to favorable games, case (b) to unfavorable games, and case (c) to fair games.

Here is the second moment martingale for the simple, symmetric random walk.

But there is another martingale that can be associated with the simple random walk, known as De Moivre's martingale and named for one of the early pioneers of probability theory, Abraham De Moivre.

The Beta-Bernoulli Process

Recall that the beta-Bernoulli process is constructed by randomizing the success parameter in a Bernoulli trials process with a beta distribution. Specifically we have a random variable $P$ that has the beta distribution with parameters $a,b\in (0,\mathrm{\infty })$, and a sequence of indicator variables $\mathit{X}=(X_1,X_2,\dots )$ such that given $P=p\in (0,1)$, $\mathit{X}$ is a sequence of independent variables with $\mathbb{P}(X_i=1)=p$ for $i\in {\mathbb{N}}_{+}$. As usual, we couch this in reliability terms, so that $X_i=1$ means success on trial $i$ and $X_i=0$ means failure. In our study of this process, we showed that the finite-dimensional distributions are given by $$\mathbb{P}(X_1=x_1,X_2=x_2,\dots ,X_n=x_n)=\frac{a^{[k]}{b}^{[n-k]}}{(a+b{)}^{[n]}},n\in {\mathbb{N}}_{+},(x_1,x_2,\dots ,x_n)\in \{0,1{\}}^n$$ where we use the ascending power notation $r^{[j]}=r(r+1)\cdots (r+j-1)$ for $r\in \mathbb{R}$ and $j\in \mathbb{N}$. Next, let $\mathit{Y}=\{Y_n:n\in \mathbb{N}\}$ denote the partial sum process associated with $\mathit{X}$, so that once again, $$Y_n=\sum _{i=1}^n{X}_i,n\in \mathbb{N}$$ Of course $Y_n$ is the number of success in the first $n$ trials and has the beta-binomial distribution defined by $$\mathbb{P}(Y_n=k)=(\genfrac{}{}{0ex}{}{n}{k})\frac{a^{[k]}{b}^{[n-k]}}{(a+b{)}^{[n]}},k\in \{0,1,\dots ,n\}$$ Now let $$Z_n=\frac{a+Y_n}{a+b+n},n\in \mathbb{N}$$ This variable also arises naturally. Let ${\mathcal{F}}_n=\sigma \{X_1,X_2,\dots ,X_n\}$ for $n\in \mathbb{N}$. Then as shown in the section on the beta-Bernoulli process, $Z_n=\mathbb{E}(X_{n+1}\mid {\mathcal{F}}_n)=\mathbb{E}(P\mid {\mathcal{F}}_n)$. In statistical terms, the second equation means that $Z_n$ is the Bayesian estimator of the unknown success probability $p$ in a sequence of Bernoulli trials, when $p$ is modeled by the random variable $P$.

Pólya's Urn Process

Recall that in the simplest version of Pólya's urn process, we start with an urn containing $a$ red and $b$ green balls. At each discrete time step, we select a ball at random from the urn and then replace the ball and add $c$ new balls of the same color to the urn. For the parameters, we need $a,b\in {\mathbb{N}}_{+}$ and $c\in \mathbb{N}$. For $i\in {\mathbb{N}}_{+}$, let $X_i$ denote the color of the ball selected on the $i$th draw, where 1 means red and 0 means green. The process $\mathit{X}=\{X_n:n\in {\mathbb{N}}_{+}\}$ is a classical example of a sequence of exchangeable yet dependent variables. Let $\mathit{Y}=\{Y_n:n\in \mathbb{N}\}$ denote the partial sum process associated with $\mathit{X}$, so that once again, $$Y_n=\sum _{i=1}^n{X}_i,n\in \mathbb{N}$$ Of course $Y_n$ is the total number of red balls selected in the first $n$ draws. Hence at time $n\in \mathbb{N}$, the total number of red balls in the urn is $a+c{Y}_n$, while the total number of balls in the urn is $a+b+cn$ and so the proportion of red balls in the urn is $$Z_n=\frac{a+c{Y}_n}{a+b+cn}$$

Processes with Independent Increments.

Our first example above concerned the partial sum process in $\mathit{X}$ associated with a sequence of independent random variables $\mathit{V}$. Such processes are the only ones in discrete time that have independent increments. That is, for $m,n\in \mathbb{N}$ with $m\le n$, $X_n-X_m$ is independent of $(X_0,X_1,\dots ,X_m)$. The random walk process in has the additional property of stationary increments. That is, the distribution of $X_n-X_m$ is the same as the distribution of $X_{n-m}-X_0$ for $m,n\in \mathbb{N}$ with $m\le n$. Let's consider processes in discrete or continuous time with these properties. Thus, suppose that $\mathit{X}=\{X_t:t\in T\}$ satisfying the basic assumptions in above relative to the filtration $\mathfrak{F}=\{{\mathcal{F}}_s:s\in T\}$. Here are the two definitions.

Processes with stationary and independent increments were studied in the chapter on Markov processes. In continuous time (with the continuity assumptions we have imposed), such a process is known as a Lévy process, named for Paul Lévy, and also as a continuous-time random walk. For a process with independent increments (not necessarily stationary), the connection with martingales depends on the mean function $m$ given by $m(t)=\mathbb{E}(X_t)$ for $t\in T$.

Compare this result with the corresponding result for the partial sum process in [6]. Suppose now that $\mathit{X}=\{X_t:t\in [0,\mathrm{\infty })\}$ is a stochastic process as above, with mean function $m$, and let $Y_t=X_t-m(t)$ for $t\in [0,\mathrm{\infty })$. The process $\mathit{Y}=\{Y_t:t\in [0,\mathrm{\infty })\}$ is sometimes called the compensated process associated with $\mathit{X}$ and has mean function 0. If $\mathit{X}$ has independent increments, then clearly so does $\mathit{Y}$. Hence the following result is a trivial corollary to our previous theorem.

Next we give the second moment martingale for a process with independent increments, generalizing the second moment martingale for a partial sum process in [7].

Of course, since the mean function is constant, $\mathit{X}$ is also a martingale. For processes with independent and stationary increments (that is, random walks), the last two theorems simplify, because the mean and variance functions simplify.

Compare this result with the corresponding result in [11] for discrete-time random walks. Our next result is the second moment martingale. Compare this with the second moment martingale in [12] for discrete-time random walks.

In discrete time, as we have mentioned several times, all of these results reduce to the earlier results for partial sum processes and random walks. In continuous time, the Poisson processes, named of course for Simeon Poisson, provides examples. The standard (homogeneous) Poisson counting process $\mathit{N}=\{N_t:t\in [0,\mathrm{\infty })\}$ with constant rate $r\in (0,\mathrm{\infty })$ has stationary, independent increments and mean function given by $m(t)=rt$ for $t\in [0,\mathrm{\infty })$. More generally, suppose that $r:[0,\mathrm{\infty })\to (0,\mathrm{\infty })$ is piecewise continuous (and non-constant). The non-homogeneous Poisson counting process $\mathit{N}=\{N_t:t\in [0,\mathrm{\infty })\}$ with rate function $r$ has independent increments and mean function given by $$m(t)={\int }_0^{t}r(s)ds,t\in [0,\mathrm{\infty })$$ The increment $N_t-N_s$ has the Poisson distribution with parameter $m(t)-m(s)$ for $s,t\in [0,\mathrm{\infty })$ with $s<t$, so the process does not have stationary increments. In all cases, $m$ is increasing, so the following results are corollaries of our general results:

We will see further examples of processes with stationary, independent increments in continuous time (and so also examples of continuous-time martingales) in our study of Brownian motion.

Likelihood Ratio Tests

Suppose that $(S,\mathcal{S},\mu )$ is a general measure space, and that $\mathit{X}=\{X_n:n\in \mathbb{N}\}$ is a sequence of independent, identically distributed random variables, taking values in $S$. In statistical terms, $\mathit{X}$ corresponds to sampling from the common distribution, which is usually not completely known. Indeed, the central problem in statistics is to draw inferences about the distribution from observations of $\mathit{X}$. Suppose now that the underlying distribution either has probability density function $g_0$ or probability density function $g_1$, with respect to $\mu$. We assume that $g_0$ and $g_1$ are positive on $S$. Of course the common special cases of this setup are

• $S$ is a measurable subset of ${\mathbb{R}}^n$ for some $n\in {\mathbb{N}}_{+}$ and $\mu ={\lambda }^n$ is $n$-dimensional Lebesgue measure on $S$.
• $S$ is a countable set and $\mu =\mathrm{\#}$ is counting measure on $S$.

The likelihood ratio test is a hypothesis test, where the null and alternative hypotheses are

• $H_0$: the probability density function is $g_0$.
• $H_1$: the probability density function is $g_1$.

The test is based on the test statistic $$L_n=\prod _{i=1}^n\frac{g_0(X_i)}{g_1(X_i)},n\in \mathbb{N}$$ known as the likelihood ratio test statistic. Small values of the test statistic are evidence in favor of the alternative hypothesis $H_1$. Here is our result.

Branching Processes

In the simplest model of a branching process, we have a system of particles each of which can die out or split into new particles of the same type. The fundamental assumption is that the particles act independently, each with the same offspring distribution on $\mathbb{N}$. We will let $f$ denote the (discrete) probability density function of the number of offspring of a particle, $m$ the mean of the distribution, and $\varphi$ the probability generating function of the distribution. Thus, if $U$ is the number of children of a particle, then $f(n)=\mathbb{P}(U=n)$ for $n\in \mathbb{N}$, $m=\mathbb{E}(U)$, and $\varphi (t)=\mathbb{E}\left(t^U\right)$ defined at least for $t\in (-1,1]$.

Our interest is in generational time rather than absolute time: the original particles are in generation 0, and recursively, the children a particle in generation $n$ belong to generation $n+1$. Thus, the stochastic process of interest is $\mathit{X}=\{X_n:n\in \mathbb{N}\}$ where $X_n$ is the number of particles in the $n$th generation for $n\in \mathbb{N}$. The process $\mathit{X}$ is a Markov chain and was studied in the section on discrete-time branching chains. In particular, one of the fundamental problems is to compute the probability $q$ of extinction starting with a single particle: $$q=\mathbb{P}(X_n=0\text{ for some }n\in \mathbb{N}\mid X_0=1)$$ Then, since the particles act independently, the probability of extinction starting with $x\in \mathbb{N}$ particles is simply $q^x$. We will assume that $f(0)>0$ and $f(0)+f(1)<1$. This is the interesting case, since it means that a particle has a positive probability of dying without children and a positive probability of producing more than 1 child. The fundamental result, you may recall, is that $q$ is the smallest fixed point of $\varphi$ (so that $\varphi (q)=q$) in the interval $[0,1]$. Here are two martingales associated with the branching process: