Conditional Probability: Other Stuff

More notes on conditional probability.

$$$$

Stuff to remember

Main page | Background

• HJ’s non measurable diagonal example

• HJ’s Leb example

• Faden (1985) nec and suff conditions; Leb example

• Pachl (1978): has Leb example too (Marczewski)

• P. Pfanzagl (1979): Conditional Distributions as Derivatives

• J. Pfanzagl (1969): On the existence of regular conditional probabilities

• Wise and Hall (Counterexamples in Probability and Real Analysis)

  • Doob example is 8.2 p. 160 (more detail) Cond Prob is not a measure and hence rcp does not exist
  • Conditional expectation need not be a smoothing operator. Ex. 8.6 p. 162

  Perhaps a little less obvious is the fact that, for a random variable $X$ on $(\mathrm{\Omega },\mathcal{F},\mathsf{P})$ and a $\sigma$-subalgebra $\mathcal{G}$ of $\mathcal{F}$, $\mathsf{E}[X\mid \mathcal{G}]$ need not be as smooth a function of $\omega$ as $X$, where the set $\mathrm{\Omega }$ is assumed to have some associated topology. Consider, for instance, the probability space given by the interval $[0,1]$, the $\sigma$-algebra $\mathcal{G}$ given by the countable and co-countable subsets of $[0,1]$, and Lebesgue measure on $\mathcal{G}$. If $X=1$, then a version of $\mathsf{E}[X\mid \mathcal{G}]$ is given by $1—1_B$, where $B$ equals the set of rationals in $[0,1]$. Hence, for this choice of $X$, even though $X$ is everywhere continuous, there exists a version of $\mathsf{E}[X\mid \mathcal{G}]$ that is everywhere discontinuous.

• Dudley (2018) on disintegration.

• Ash (2014)

• Parthasarathy (1967) on disintegration

• Dellacherie and Meyer (1978)

  • Univ mesaurable (Radon) sections 31-II
  • Conditioning 36 and Disintegration theorem cf. Pollard (actual p. 84)

• Meaning of conditional expectation per (Rogers and Williams 1994, p. II.39)

  An experiment has been performed. The only information available to you regarding which sample point $\omega$ has been chosen is the set of values $Z(\omega )$ for every $\mathcal{G}$-measurable random variable $Z$, or, equivalently, the values $1_G(\omega )$ for every $G\in \mathcal{G}$. Then $Y(\omega )=\mathsf{E}[X\mid \mathcal{G}](\omega )$ is regarded as (almost surely equal to) the “expected value of $X(\omega )$ given this information”.

  Note that if $\mathcal{G}$ is the trivial $\sigma$-algebra $\{\mathrm{\varnothing },\Omega \}$ (which contains no information) then $\mathsf{E}[X\mid \mathcal{G}](\omega )=\mathsf{E}[X]$ for all $\omega$.

Weird Sub sigma algebras

• Countable or co-countable sets [SM: = generated by the points(!)]
• Sets of measure zero or 1
• Sets of measure zero as a subset of the completed sigma algebra
• Tail sigma algebra (cf. Jacod and Shiryaev (2013) (compensators for exponential process)

---

Tables

---

LLM Material

Background on Measure Theory

Monotone Class Arguments

The monotone class argument is a powerful technique in measure theory and probability theory, used to prove that certain classes of sets or functions are closed under various operations. Here’s a brief overview:

Definition and Setup

A monotone class is a collection of sets (or functions) that is closed under certain limits: 1. Increasing limits: If $A_n\in \mathcal{C}$ and $A_n\uparrow A$ (i.e., $A_1\subseteq A_2\subseteq \cdots$ and $A=\bigcup _{n=1}^{\mathrm{\infty }}{A}_n$), then $A\in \mathcal{C}$. 2. Decreasing limits: If $A_n\in \mathcal{C}$ and $A_n\downarrow A$ (i.e., $A_1\supseteq A_2\supseteq \cdots$ and $A=\bigcap _{n=1}^{\mathrm{\infty }}{A}_n$), then $A\in \mathcal{C}$.

Monotone Class Theorem

The monotone class theorem states that if $\mathcal{A}$ is an algebra of sets (closed under finite unions and intersections) in a measurable space $(X,\mathrm{\Sigma })$, and $\mathcal{M}$ is a monotone class containing $\mathcal{A}$, then $\mathcal{M}$ contains the $\sigma$-algebra generated by $\mathcal{A}$.

Proof Technique Using Monotone Classes

To prove that a certain property holds for all sets in a $\sigma$-algebra $\mathrm{\Sigma }$, one can often proceed as follows:

1. Identify a class $\mathcal{C}$: Define a class of sets (or functions) for which the desired property holds.
2. Verify initial conditions: Show that this class is an algebra.
3. Verify closure properties: Show that this class is a monotone class.

By the monotone class theorem, if the class is an algebra and a monotone class, it must contain the $\sigma$-algebra generated by the algebra, thus the property holds for all sets in this $\sigma$-algebra.

Example Application

Proving that a measure is unique:

Suppose $\mu$ and $\nu$ are two measures on a measurable space $(X,\mathrm{\Sigma })$ that agree on an algebra $\mathcal{A}$. To show $\mu =\nu$ on $\sigma (\mathcal{A})$:

1. Define $\mathcal{C}$: Let $\mathcal{C}=\{A\in \mathrm{\Sigma }:\mu (A)=\nu (A)\}$.
2. Show $\mathcal{A}\subseteq \mathcal{C}$: By assumption, $\mu (A)=\nu (A)$ for all $A\in \mathcal{A}$.
3. Show $\mathcal{C}$ is an algebra: Check closure under finite unions and intersections.
4. Show $\mathcal{C}$ is a monotone class: Check closure under increasing and decreasing limits.

By the monotone class theorem, $\mathcal{C}$ contains $\sigma (\mathcal{A})$, hence $\mu =\nu$ on $\sigma (\mathcal{A})$.

The monotone class argument is a crucial tool for establishing properties on $\sigma$-algebras by first verifying them on more manageable classes of sets, leveraging closure properties to extend results.

Carathéodory’s Extension Theorem

Carathéodory’s Extension Theorem is a fundamental result in measure theory. It provides a method for constructing a measure on a $\sigma$-algebra from a pre-measure defined on an algebra. Here is a detailed explanation:

Carathéodory’s Extension Theorem

Statement

Let $\mathcal{A}$ be an algebra of subsets of a set $X$ and let ${\mu }_0:\mathcal{A}\to [0,\mathrm{\infty }]$ be a pre-measure (a function satisfying the conditions of a measure, but only defined on $\mathcal{A}$).

Then there exists a measure $\mu$ on the $\sigma$-algebra $\sigma (\mathcal{A})$ generated by $\mathcal{A}$ such that $\mu$ extends ${\mu }_0$. This means that $\mu$ restricted to $\mathcal{A}$ equals ${\mu }_0$.

Additionally, if ${\mu }_0$ is $\sigma$-finite (i.e., $X$ can be written as a countable union of sets in $\mathcal{A}$ with finite ${\mu }_0$-measure), then $\mu$ is uniquely determined by ${\mu }_0$.

Construction Outline

1. Define Outer Measure: Construct an outer measure ${\mu }^{\ast }$ on $X$ from ${\mu }_0$. $${\mu }^{\ast }(E)=inf\{\sum _{i=1}^{\mathrm{\infty }}{\mu }_0(A_i):E\subseteq \bigcup _{i=1}^{\mathrm{\infty }}{A}_i,A_i\in \mathcal{A}\}.$$

2. Carathéodory’s Criterion: Define the $\sigma$-algebra $\mathcal{M}$ of ${\mu }^{\ast }$-measurable sets: $$\mathcal{M}=\{E\subseteq X:{\mu }^{\ast }(A)={\mu }^{\ast }(A\cap E)+{\mu }^{\ast }(A\cap E^c)\text{ for all }A\subseteq X\}.$$

3. Measure on $\mathcal{M}$: The restriction of ${\mu }^{\ast }$ to $\mathcal{M}$ is a measure, denoted by $\mu$.

4. Extension: Show that $\mathcal{M}=\sigma (\mathcal{A})$ and $\mu$ restricted to $\mathcal{A}$ coincides with ${\mu }_0$.

Importance and Applications

Carathéodory’s Extension Theorem is crucial because it allows the construction of measures on $\sigma$-algebras, which are necessary for integration and probability theory. This theorem ensures that starting with a simple pre-measure on an algebra, one can extend it to a complete measure on a larger, more useful $\sigma$-algebra.

Example

A common application is the construction of the Lebesgue measure on $\mathbb{R}$:

1. Algebra: Start with the algebra of finite unions of intervals.
2. Pre-measure: Define the pre-measure as the length of the interval.
3. Extension: Use Carathéodory’s Extension Theorem to extend this pre-measure to the Borel $\sigma$-algebra, resulting in the Lebesgue measure.

Carathéodory’s Extension Theorem provides the foundation for developing measures from simpler constructs, making it a cornerstone of modern measure theory.

References

Ash, R. B. (2014). Real analysis and probability: Probability and mathematical statistics: A series of monographs and textbooks. Academic press.

Dellacherie, C., & Meyer, P.-A. (1978). Probabilities and potential. Hermann.

Dudley, R. M. (2018). Real analysis and probability. Chapman; Hall/CRC.

Faden, A. M. (1985). The Existence of Regular Conditional Probabilities: Necessary and Sufficient Conditions. The Annals of Probability, 13(1), 288–298. http://projecteuclid.org/euclid.aop/1176996548

Jacod, J., & Shiryaev, A. (2013). Limit theorems for stochastic processes (Vol. 288). Springer Science & Business Media.

Pachl, J. K. (1978). Disintegration and compact measures. https://doi.org/10.7146/math.scand.a-11771

Parthasarathy, K. R. (1967). Probability Measures on Metric Spaces. Academic Press.

Pfanzagl, J. (1969). On the existence of regular conditional probabilities. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 11(3), 244–256. https://doi.org/10.1007/BF00536383

Pfanzagl, P. (1979). Conditional Distributions as Derivatives. The Annals of Probability, 7(6), 347–370. https://doi.org/10.1214/aop/1176994897

Rogers, L. C. G., & Williams, D. (1994). Diffusions, Markov Processes, and Martingales, Volume 1: Foundations (Second.). John Wiley; Sons.