Conditional Probability: Other Stuff
\[ \def\E{\mathsf E} \def\F{\mathscr F} \def\G{\mathscr G} \def\A{\mathscr A} \def\EE{\mathscr E} \def\S{\mathscr S} \def\B{\mathcal B} \def\P{\mathsf P} \def\Q{\mathsf Q} \def\R{\mathbb R} \def\px{\phantom{X}} \]
Stuff to remember
HJ’s non measurable diagonal example
HJ’s Leb example
Faden (1985) nec and suff conditions; Leb example
Pachl (1978): has Leb example too (Marczewski)
P. Pfanzagl (1979): Conditional Distributions as Derivatives
J. Pfanzagl (1969): On the existence of regular conditional probabilities
Wise and Hall (Counterexamples in Probability and Real Analysis)
- Doob example is 8.2 p. 160 (more detail) Cond Prob is not a measure and hence rcp does not exist
- Conditional expectation need not be a smoothing operator. Ex. 8.6 p. 162
Perhaps a little less obvious is the fact that, for a random variable \(X\) on \((\Omega,\F,\P)\) and a \(\sigma\)-subalgebra \(\G\) of \(\F\), \(\E[X \mid \G]\) need not be as smooth a function of \(ω\) as \(X\), where the set \(\Omega\) is assumed to have some associated topology. Consider, for instance, the probability space given by the interval \([0, 1]\), the \(\sigma\)-algebra \(\G\) given by the countable and co-countable subsets of \([0, 1]\), and Lebesgue measure on \(\G\). If \(X = 1\), then a version of \(\E[X \mid \G]\) is given by \(1 — 1_B\), where \(B\) equals the set of rationals in \([0, 1]\). Hence, for this choice of \(X\), even though \(X\) is everywhere continuous, there exists a version of \(\E[X \mid \G]\) that is everywhere discontinuous.
Dudley (2018) on disintegration.
Ash (2014)
Parthasarathy (1967) on disintegration
Dellacherie and Meyer (1978)
- Univ mesaurable (Radon) sections 31-II
- Conditioning 36 and Disintegration theorem cf. Pollard (actual p. 84)
Meaning of conditional expectation per (Rogers and Williams 1994, p. II.39)
An experiment has been performed. The only information available to you regarding which sample point \(ω\) has been chosen is the set of values \(Z(ω)\) for every \(\G\)-measurable random variable \(Z\), or, equivalently, the values \(1_G(ω)\) for every \(G\in\G\). Then \(Y(ω)=\E[X\mid\G](ω)\) is regarded as (almost surely equal to) the “expected value of \(X(ω)\) given this information”.
Note that if \(\G\) is the trivial \(\sigma\)-algebra \(\{\emptyset, Ω\}\) (which contains no information) then \(\E[X\mid\G](ω)=\E[X]\) for all \(ω\).
Weird Sub sigma algebras
- Countable or co-countable sets [SM: = generated by the points(!)]
- Sets of measure zero or 1
- Sets of measure zero as a subset of the completed sigma algebra
- Tail sigma algebra (cf. Jacod and Shiryaev (2013) (compensators for exponential process)
Tables
LLM Material
Background on Measure Theory
Monotone Class Arguments
The monotone class argument is a powerful technique in measure theory and probability theory, used to prove that certain classes of sets or functions are closed under various operations. Here’s a brief overview:
Definition and Setup
A monotone class is a collection of sets (or functions) that is closed under certain limits: 1. Increasing limits: If \(A_n \in \mathcal{C}\) and \(A_n \uparrow A\) (i.e., \(A_1 \subseteq A_2 \subseteq \cdots\) and \(A = \bigcup_{n=1}^{\infty} A_n\)), then \(A \in \mathcal{C}\). 2. Decreasing limits: If \(A_n \in \mathcal{C}\) and \(A_n \downarrow A\) (i.e., \(A_1 \supseteq A_2 \supseteq \cdots\) and \(A = \bigcap_{n=1}^{\infty} A_n\)), then \(A \in \mathcal{C}\).
Monotone Class Theorem
The monotone class theorem states that if \(\mathcal{A}\) is an algebra of sets (closed under finite unions and intersections) in a measurable space \((X, \Sigma)\), and \(\mathcal{M}\) is a monotone class containing \(\mathcal{A}\), then \(\mathcal{M}\) contains the \(\sigma\)-algebra generated by \(\mathcal{A}\).
Proof Technique Using Monotone Classes
To prove that a certain property holds for all sets in a \(\sigma\)-algebra \(\Sigma\), one can often proceed as follows:
- Identify a class \(\mathcal{C}\): Define a class of sets (or functions) for which the desired property holds.
- Verify initial conditions: Show that this class is an algebra.
- Verify closure properties: Show that this class is a monotone class.
By the monotone class theorem, if the class is an algebra and a monotone class, it must contain the \(\sigma\)-algebra generated by the algebra, thus the property holds for all sets in this \(\sigma\)-algebra.
Example Application
Proving that a measure is unique:
Suppose \(\mu\) and \(\nu\) are two measures on a measurable space \((X, \Sigma)\) that agree on an algebra \(\mathcal{A}\). To show \(\mu = \nu\) on \(\sigma(\mathcal{A})\):
- Define \(\mathcal{C}\): Let \(\mathcal{C} = \{ A \in \Sigma : \mu(A) = \nu(A) \}\).
- Show \(\mathcal{A} \subseteq \mathcal{C}\): By assumption, \(\mu(A) = \nu(A)\) for all \(A \in \mathcal{A}\).
- Show \(\mathcal{C}\) is an algebra: Check closure under finite unions and intersections.
- Show \(\mathcal{C}\) is a monotone class: Check closure under increasing and decreasing limits.
By the monotone class theorem, \(\mathcal{C}\) contains \(\sigma(\mathcal{A})\), hence \(\mu = \nu\) on \(\sigma(\mathcal{A})\).
The monotone class argument is a crucial tool for establishing properties on \(\sigma\)-algebras by first verifying them on more manageable classes of sets, leveraging closure properties to extend results.
Carathéodory’s Extension Theorem
Carathéodory’s Extension Theorem is a fundamental result in measure theory. It provides a method for constructing a measure on a \(\sigma\)-algebra from a pre-measure defined on an algebra. Here is a detailed explanation:
Carathéodory’s Extension Theorem
Statement
Let \(\mathcal{A}\) be an algebra of subsets of a set \(X\) and let \(\mu_0 : \mathcal{A} \to [0, \infty]\) be a pre-measure (a function satisfying the conditions of a measure, but only defined on \(\mathcal{A}\)).
Then there exists a measure \(\mu\) on the \(\sigma\)-algebra \(\sigma(\mathcal{A})\) generated by \(\mathcal{A}\) such that \(\mu\) extends \(\mu_0\). This means that \(\mu\) restricted to \(\mathcal{A}\) equals \(\mu_0\).
Additionally, if \(\mu_0\) is \(\sigma\)-finite (i.e., \(X\) can be written as a countable union of sets in \(\mathcal{A}\) with finite \(\mu_0\)-measure), then \(\mu\) is uniquely determined by \(\mu_0\).
Construction Outline
Define Outer Measure: Construct an outer measure \(\mu^*\) on \(X\) from \(\mu_0\). \[ \mu^*(E) = \inf \left\{ \sum_{i=1}^\infty \mu_0(A_i) : E \subseteq \bigcup_{i=1}^\infty A_i, A_i \in \mathcal{A} \right\}. \]
Carathéodory’s Criterion: Define the \(\sigma\)-algebra \(\mathcal{M}\) of \(\mu^*\)-measurable sets: \[ \mathcal{M} = \{ E \subseteq X : \mu^*(A) = \mu^*(A \cap E) + \mu^*(A \cap E^c) \text{ for all } A \subseteq X \}. \]
Measure on \(\mathcal{M}\): The restriction of \(\mu^*\) to \(\mathcal{M}\) is a measure, denoted by \(\mu\).
Extension: Show that \(\mathcal{M} = \sigma(\mathcal{A})\) and \(\mu\) restricted to \(\mathcal{A}\) coincides with \(\mu_0\).
Importance and Applications
Carathéodory’s Extension Theorem is crucial because it allows the construction of measures on \(\sigma\)-algebras, which are necessary for integration and probability theory. This theorem ensures that starting with a simple pre-measure on an algebra, one can extend it to a complete measure on a larger, more useful \(\sigma\)-algebra.
Example
A common application is the construction of the Lebesgue measure on \(\mathbb{R}\):
- Algebra: Start with the algebra of finite unions of intervals.
- Pre-measure: Define the pre-measure as the length of the interval.
- Extension: Use Carathéodory’s Extension Theorem to extend this pre-measure to the Borel \(\sigma\)-algebra, resulting in the Lebesgue measure.
Carathéodory’s Extension Theorem provides the foundation for developing measures from simpler constructs, making it a cornerstone of modern measure theory.