We present some additional results from Measure Theory (monotone class theorem), Probability (martingales and CLT), Functional Analysis (maximal functions) and Topology (weak compactness).
Monoton Class Theorem
A collection of subsets ${\cal P}\sbe{\cal P}(\O)$ is called a $\pi$-system, if for all $A,B\in{\cal P}$: $A\cap B\in{\cal P}$. A collection of subsets ${\cal L}\sbe{\cal P}(\O)$ is called a $\l$-system if
- $\O\in{\cal L}$.
- $A,B\in{\cal L}$ and $B\sbe A$ imply: $A\sm B\in{\cal L}$.
- $A_n\in{\cal L}$ and $A_n\uar$ imply: $\bigcup A_n\in{\cal L}$.
Dynkin's $\pi$-$\l$-Theorem asserts that if a $\l$-system ${\cal L}$ comprises a $\pi$-system ${\cal P}$, then $\s({\cal P})\sbe{\cal L}$. A typical application is
For $A_j\in{\cal P}_j$, $j=2,\ldots,n$ put
$$
{\cal L}\colon=\{A\in\s({\cal P}_1):\,
\P(A\cap A_2\cap\ldots\cap A_n)
=\P(A)\P(A_2)\cdots\P(A_n)\}~.
$$
Then ${\cal L}$ is a $\l$-system and by Dynkin's $\pi$-$\l$-Theorem: ${\cal L}=\s({\cal P}_1)$. Iterating the argument proves the assertion.
$\proof$
Put ${\cal L}\colon=\{A:\,I_A\in E\}$, then this is a $\l$-system and by Dynkin's $\pi$-$\l$-Theorem $E$ contains all simple functions, i.e. finite linear combinations of indicators $I_A$, $A\in\s({\cal P})$. By a result from measure theory every non negative, bounded measurable function is the uniform limit of a sequence of increasing, simple functions; hence $E$ contains all bounded measurable functions.
$\eofproof$
Martingales and Submartingales
Let $(\O,\F,\P)$ be a probability space and $\F_n\sbe\F$ a so called filtration, i.e. an increasing sequence of $\s$-algebras. A sequence of random variables $X_n$ is called a martingale (submartingale) if $X_n$ is $\F_n$-measurable and integrable and
$$
\E(X_{n+1}|\F_n)=X_n
\quad(\E(X_{n+1}|\F_n)\geq X_n)~.
$$
Given a filtration $\F_n$ and an integrable random variable $X$, the sequence $X_n\colon=\E(X|\F_n)$ is the standard example of a martingale. We will see that if $X$ is measurable with respect to the $\s$-algebra generated by $\bigcup\F_n$, then $X_n$ converges $\P$-a.s (cf. theorem) and in $L_1(\P)$ (cf. proposition) to $X$.
Suppose $X$ is integrable and $\vp:\R\rar\R$ is convex such that $\vp(X)$ is integrable. Since $\vp$ is convex there is an $\F^\prime$-measurable random variable $Z$ (in case $\vp$ is differentiable $Z=\vp^\prime(\E(X|\F^\prime))$) such that
$$
\vp(X)-\vp(\E(X|\F^\prime))
\geq Z(X-\E(X|\F^\prime)
$$
For any $A\in\F^\prime$ put $A_n\colon=A\cap[|Z|\leq n]$, then
$\E(\vp(X);A_n)\geq\E(\vp(\E(X|\F^\prime));A_n)$ and letting $n\to\infty$ we conclude that $\P$-a.s.:
$$
\vp(\E(X|\F^\prime))
\leq\E(\vp(X)|\F^\prime)~.
$$
This is the conditional version of Jensen's inequality. Now if $(M_n,\F_n)$ is a martingale, $\vp$ convex such that $\vp(M_n)$ is integrable, then $(\vp(M_n),\F_n)$ is a submartingale:
$$
\E(\vp(M_{n+1})|\F_n)
\geq\vp(\E(M_{n+1})|\F_n))
=\vp(M_n)~.
$$
Upcrossing
Suppose $(X_n,\F_n)$ is a submartingale and $a < b$ are real numbers. Define so called stopping times
\begin{eqnarray*}
S_1&\colon=&\inf\{n\geq0:\,X_n\leq a\}\\
T_k&\colon=&\inf\{n > S_k:\,X_n\geq b\}\\
S_{k+1}&\colon=&\inf\{n > T_k:\,X_n\leq a\}~.
\end{eqnarray*}
So $S_1$ is the first time the sequence $X_0,X_1,\ldots$ falls below $a$, $T_k$ is the first time after $S_k$ the sequence oversteps $b$ and $S_{k+1}$ is the first time after $T_k$ the sequence $X_0,X_1,\ldots$ falls below $a$. Finally let us put for $n\geq0$:
$$
U_n^{a,b}(X)\colon=\max\{k:\,T_k\leq n\},
$$
i.e. this is up to time $n$ the number of times the sequence $X_0,X_1,\ldots$ has crossed from some value smaller than $a$ to some value larger than $b$ - that's called the number of upcrossings.
$\proof$
Since
$$
[X_n\mbox{ converges}]
=\bigcup_{r1 < r_2\in\Q}
[\liminf X_n < r_1 < r_2 <\limsup X_n],
$$
it suffices to prove that for all $r_1 < r_2\in\Q$:
$$
\P(\liminf X_n < r_1 < r_2 < \limsup X_n)=0.
$$
If $\P(\liminf X_n < r_1 < r_2 < \limsup X_n) > 0$, then the sequence $X_0,X_1,\ldots$ upcrosses the interval $[r_1,r_2]$ infinitely often on a set of strictly positive measure. Hence $\E U=\infty$. On the other hand $U\colon=\sup U_n$ is the limit of the increasing sequence $U_n$ and by the Upcrossing Lemma
$$
(r_2-r_1)\E U_n\leq\E(X_n-r_1)^+
\leq\sup\E X_n^++|r_1|
$$
It follows $U$ is integrable. Consequently $X_n$ must converge $\P$-a.s. to some random variable $X$. Finally
$$
\E|X_n|
=\E(2X_n^+-X_n)
\leq2\sup\E X_n^+-\E X_n~.
$$
and thus by Fatou's lemma $X$ must be integrable.
$\eofproof$
In order to verify a.s. convergence it's thus obvious to study the maximal function
$X^*\colon=\sup_n|X_n|$.
Doob's Inequalities
For any squence of random variables $X_n$ let us put
$$
X_n^*\colon=\sup_{k\leq n}X_k
$$
$\proof$
Define $T\colon=\inf\{k:X_k > \e\}$, then $[T=k]=[X_{k-1}^*\leq\e,X_k > \e]\in\F_k$ and
\begin{eqnarray*}
\e\P(X_n^* > \e)
&=&\e\P(T\leq n)
=\sum_{k=1}^n\e\P(T=k)\\
&\leq&\sum_{k=1}^n\E(X_k;T=k)
\leq\sum_{k=1}^n\E(X_n;T=k)
=\E(X_n;T\leq n)~.
\end{eqnarray*}
$\eofproof$
$Y_n\colon=|X_n|$ is a non-negative submartingale and by Doob's inequality, Fubini and Hölder's inequality:
\begin{eqnarray*}
\E Y_n^{*p}
&=&\int_0^\infty pr^{p-1}\P(Y_n^* > r)\,dr
\leq p\int_0^\infty r^{p-2}\int_{[Y_n^* > r]}Y_n\,d\P dr\\
&=&p\E\int_0^{Y_n^*}Y_n\,r^{p-2}\,dr
=\frac p{p-1}\E(Y_n^{*(p-1)}Y_n)
\leq\frac p{p-1}(\E Y_n^{*(p-1)q})^{1/q}(\E Y_n^p)^{1/p}
\end{eqnarray*}
and since $q=p/(p-1)$, the conclusion follows.
$\eofproof$
A subset $A$ of $L_1(\P)$ is said to be uniformly integrable if
$$
\lim_{r\to\infty}\sup_{X\in A}\E(|X|;|X| > r)=0~.
$$
Of course, any finite set of integrable random variables is uniformly integrable. Uniformly integrable subsets of $L_1(\P)$ are bounded; in fact they are just the weakly relatively compact subsets of $L_1(\P)$, cf. Dunford-Pettis Theorem. If $X_n$ is a bounded sequence in $L_p(\P)$ for some $p > 1$, then by Hölder's and Chebyshev's inequality:
$$
\E(|X_n|;|X_n| > r)
\leq\norm{X_n}_p\P(|X_n| > r)^{1/q}
\leq\norm{X_n}_p r^{-p/q}(\E|X_n|^p){1/q}
=r^{-1}\norm{X_n}_p^{1+p/q}
$$
Hence bounded subsets in $L_p(\P)$, $p > 1$, are uniformly integrable. More generally, we have
$\proof$
1. $\Rar$ 2.: As uniformly integrable sets are bounded, we conclude that: $\sup_n\E|X_n| < \infty$. Hence by Fatou's lemma: $\E|X| < \infty$. For $a > 0$ put $f_a(x)=x$ if $|x|\leq a$ and $0$ otherwise. It follows that
\begin{eqnarray*}
\E|X_n-X|
&\leq&\E|f_a(X_n)-f_a(X)|+\E|X_n-f_a(X_n)|+\E|X-f_a(X)|\\
&\leq&\E|f_a(X_n)-f_a(X)|+\sup_n\E(|X_n|;|X_n| > a)+\E(|X|;|X| > a)~.
\end{eqnarray*}
The first term converges to $0$ by bounded convergence for all $a > 0$; the second and the third term can be made arbitrarily small by choosing $a$ sufficiently large.
2. $\Rar$ 3.: $|\E|X_n|-\E|X||\leq\E|X_n-X|$.
3. $\Rar$ 1.: By bounded convergence we have: $\lim_n\E f_a(|X_n|)=\E f_a(|X|)$ and thus for $\e > 0$ and all $n\geq n(\e)$:
$$
\E(|X_n|;|X_n| > a)
=\E|X_n|-\E f_a(|X_n|)
\leq\E|X|-\E f_a(|X|)+2\e~.
$$
For sufficiently large $a$ the right hand side becomes smaller than $3\e$. Since finite sets are uniformly integrable we conclude that $\{X_n:n\in\N\}$ is uniformly integrable.
$\eofproof$
Thus a uniformly integrable submartingale $(X_n,\F_n)$ converges in $L_1(\P)$ to some integrable random variable $X$. If in addition $X_n$ is a uniformly integrable martingale, then $X_n=\E(X|\F_n)$.
Reverse martingales
Putting $Y_{-n}\colon=X_n$ and $\F_n^\prime=\F_{-n}$ we see that $(X_n,\F_n)$, $n\geq0$ is a reverse submartingal iff $(Y_n,\F_n^\prime)$, $n\leq0$ is a submartingale. Probably the most prominent example of a reverse martingale is the sequence of arithmetic means
$$
M_n\colon=\frac1nS_n\colon=\frac1n(X_1+\cdots+X_n)
$$
of i.i.d. random variables $X_1,X_2,\ldots$ with respect to $\F_n\colon=\s(S_n,X_{n+1},\ldots)$: indeed by symmetry we have for all $j,k=1,\ldots,n+1$: $\E(X_j|\F_{n+1})=\E(X_k|\F_{n+1})$ and since $\E(S_{n+1}|\F_{n+1})=S_{n+1}$ we conclude that $\E(X_j|\F_{n+1})=M_{n+1}$. It follows that
$$
\E(M_n|\F_{n+1})=\frac1n\sum_{j=1}^n\E(X_j|\F_{n+1})=M_{n+1}~.
$$
$\proof$
The sequence $\E X_n$ decreases to some limit $\a\in\R$. For $\e > 0$ we can find an index $m$. such that for all $n\geq m$ gilt: $\E X_m\geq\E X_n > \E X_m-\e$. For any $r > 0$ we have:
\begin{eqnarray*}
\E(|X_n|;|X_n| > r)
&=&\E(X_n;X_n > r)+\E(-X_n;X_n < -r)\\
&=&\E(X_n;X_n > r)-\E X_n+\E(X_n;X_n\geq-r)\\
&\leq&\E(X_m;X_n > r)-\E X_m+\e+\E(X_m;X_n\geq-r)\\
&=&\E(X_m;X_n > r)+\e+\E(-X_m;X_n < -r)
=\E(|X_m|;|X_n| > r)+\e
\end{eqnarray*}
Finally
$$
\E|X_n|
=\E X_n^+-\E X_n^-
=2\E X_n^+-\E X_n
\leq-\a+2\E X_1^+
=\colon\b
$$
and thus $\P(|X_n| > r)\leq t^{-1}\b$, i.e. $X_n$ is uniformly integrable.
Put $Y_{-j}\colon=X_j$, then we infer from the Upcrossing Lemma applied to the submartingal $Y_{-n},\ldots,Y_{0}$:
$$
U_n^{a,b}(Y)\leq\frac{\E(X_0-a)^+}{b-a}~.
$$
This shows that $Y_{-n}$ converges both $\P$-a.s. and in $L_1(\P)$ to $X$. Since $X$ is measurable with respect to all $\s$-algebren $\F_n$, it's measurable with respect to $\bigcap\F_n$.
If $X_n$ is a reverse martingale, then for all $A\in\bigcap\F_n$:
$$
\E(X_0;A)=\lim_n\E(X_n;A)=\E(X;A),
$$
i.e. $X=\E(X_0|\bigcap\F_n)$.
$\eofproof$
Optional stopping
$\F_T$ is a $\s$-algebra and every stopping time $T$ is $\F_T$-measurable. Moreover, if $T\leq n$, then $\F_T\sbe\F_n$.
Let $X_n:\O\rar S$ be a sequence of $\F_n$-adapted random variables (i.e. for all $n$ $X_n$ is $\F_n$-measurable) and $A\in\B(S)$. Then $T\colon=\inf\{n\geq0:\,X_n\in A\}$ is a stopping time, for
$$
[T\leq n]=\bigcup_{j=0}^n[X_n\in A]\in\F_n~.
$$
Moreover the random variable $X_T:\O\rar S$ is $\F_T$-measurable, because for all $B\in\B(S)$:
$$
[X_T\in B]\cap[T\leq n]=\bigcup_{j=0}^n[X_j\in B,T=j]\in\F_n
$$
Let $S$, $T$, $T_n$ be stopping times. Then $S\wedge T$, $S\vee T$, $S+T$, $\sup T_n$, $\inf T_n$, $\limsup T_n$ and $\liminf T_n$ are stopping times.
For $n\geq0$ we have: $[S+T=n]=\bigcup_{j=0}^n[S=j,T=n-j]$.
Suppose $(X_n,\F_n,\P^x)$ is a Markov chain in the Polish space $S$ and $T$ a (bounded) stopping times with respect to $\F_n$. For any bounded measurable $f:S\rar\R$ we have by the Markov property and the fact that $[T=m]\cap\F_T=[T=m]\cap\F_m$ (cf. exam):
\begin{eqnarray}
\E^x(f(X_{T+n})|\F_T)
&=&\sum_m\E^x(f(X_{m+n})|\F_T)I_{[T=m]}\\
&=&\sum_m\E^x(f(X_{m+n})|\F_m)I_{[T=m]}
=\sum_mP^nf(X_m)I_{[T=m]}
=P^nf(X_T)
=\E^{X_T}f(X_n)~.
\end{eqnarray}
This is called the strict Markov property. There is also an extended version of this property: define $\Theta_T$ by $X_n\circ\Theta_T=X_{T+n}$ for all $n\in\N_0$, then for any $\F_\infty^X$-measurable and bounded function $F:\O\rar\R$:
$$
\E^x(F\circ\Theta_T|\F_T)=\E^{X_T}F
$$
where the right hand side is $f(X_T)$ for $f(x)\colon=\E^xF$.