Probability Theory Fundamentals › Sample Spaces, Events, and Probability Axioms

Before we can assign probabilities, we need a probability space $(\Omega,\mathcal{F},P)$: a sample space $\Omega$, a sigma-algebra $\mathcal{F}$ of measurable events, and a probability measure $P$ on those events. This framework begins with three foundational concepts for discussing randomness.

The sample space contains every outcome the chosen model permits, including unlikely or undesirable ones. If an outcome is missing from that space, the model cannot assign it a probability. Physical scenarios outside the model remain a limitation of the model; listing its permitted outcomes does not establish that it describes every real-world possibility.

Consider rolling a six-sided die. The sample space is $\Omega = \{1, 2, 3, 4, 5, 6\}$, listing every possible result. For a stock's daily return, the sample space might be $\Omega = [-1, \infty)$, since returns cannot be less than -100% but can, in principle, be arbitrarily large (though extreme returns are rare). Notice how the nature of the random experiment determines the appropriate sample space: a finite set for the die, an infinite continuum for stock returns. Choosing the right sample space is the first modeling decision we make, and it shapes all subsequent analysis.

Events emerge from our questions about uncertain situations. While the sample space contains raw outcomes, events package those outcomes into meaningful categories that correspond to questions we want to answer. The event "rolling an even number" is the subset $\{2, 4, 6\}$, which collects all the individual outcomes where an even number appears. The event "stock return exceeds 5%" contains every possible return value greater than 0.05. By defining events as subsets, we gain access to the powerful machinery of set theory for combining and manipulating them.

Events can be combined using set operations. If $A$ is "stock rises" and $B$ is "volume is high," then:

• $A \cup B$ is "stock rises OR volume is high (or both)"
• $A \cap B$ is "stock rises AND volume is high"
• $A^c$ is "stock does not rise"

These set operations allow us to build complex events from simpler ones, mirroring how we naturally combine conditions in financial analysis. When a portfolio manager asks "what's the probability that either the market rallies or volatility spikes?" they're implicitly using the union operation. The intersection operation captures joint occurrences, while the complement captures the negation of an event.

With sample spaces and events defined, we need rules for assigning probabilities to events. Andrey Kolmogorov formalized probability theory in 1933 with three standard axioms that any probability measure $P$ must satisfy. They capture basic consistency requirements while leaving room for many different probability measures.

1. Non-negativity: $P(A) \geq 0$ for any event $A$
2. Normalization: $P(\Omega) = 1$ (something must happen)
3. Countable additivity: For mutually exclusive events $A_1, A_2, \ldots$,

The first axiom says probabilities cannot be negative. The second normalizes the scale by assigning $P(\Omega)=1$, while $P(\varnothing)=0$ follows from the axioms. A nontrivial event can still have probability 1 or 0 without being logically certain or impossible; such events occur almost surely or have probability zero. The third axiom says that for mutually exclusive events, the probability of at least one occurring equals the sum of their individual probabilities.

Together with the measurable space and the surrounding mathematics, these axioms define a probability measure and yield the rules used in this chapter. Several useful properties follow immediately:

• Complement rule: $P(A^c) = 1 - P(A)$
• Probability bounds: $0 \leq P(A) \leq 1$
• Addition rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

The complement rule follows from the fact that $A$ and $A^c$ are mutually exclusive and their union is the entire sample space: $P(A) + P(A^c) = P(\Omega) = 1$. The probability bounds follow from the complement rule combined with non-negativity. The addition rule requires subtracting $P(A \cap B)$ to avoid double-counting outcomes that belong to both events.

The addition rule accounts for double-counting when events overlap. If the probability of a stock rising is 0.55 and the probability of high volume is 0.30, we cannot simply add these to find the probability of either occurring; we must subtract the probability of both happening together. This correction is essential whenever we combine events that share common outcomes.