Probability Theory Fundamentals › Random Variables

While sample spaces and events provide the conceptual foundation, we typically work with random variables that map outcomes to numbers. This mapping enables mathematical operations like addition, multiplication, and the computation of averages. The transition from abstract outcomes to numerical values makes probability theory quantitative; it lets us apply calculus, linear algebra, and optimization to uncertainty.

The definition of a random variable as a function might initially seem abstract, but it captures a natural idea: we take the raw outcomes of a random experiment and translate them into numbers that we can analyze mathematically. The word "variable" reflects that the numerical value varies depending on which outcome occurs. The word "random" indicates that we don't know in advance which outcome will occur.

When we say "let $X$ be the daily return of a stock," we mean $X$ is a function that, for each possible market scenario $\omega$, produces a number representing that day's return. Before the market closes, $X$ is random because we do not know which $\omega$ will occur. After closing, we observe a specific realization $x = 0.023$ (a 2.3% return). The distinction between $X$ and its realized value $x$ determines which tools apply. Before observation, we work with probabilities and expectations; after observation, we work with data.

A discrete random variable takes values from a countable set, such as integers or a finite list. Discrete random variables arise when outcomes fall into distinct categories or when we count occurrences of events. The probability mass function (PMF) specifies the probability of each possible value, giving us a complete description of how probability is distributed across the possible outcomes.

The PMF answers the most direct question we can ask about a discrete random variable: "What's the probability that $X$ takes this particular value?" By specifying these probabilities for every possible value, the PMF provides complete information about the random variable's behavior. The subscript $X$ in $p_X(x)$ reminds us which random variable we're describing, since different random variables have different PMFs.

Consider a simplified credit rating model where a bond can be in one of three states at year-end:

The probabilities sum to 1 because exactly one outcome must occur: this bond will be upgraded, unchanged, or downgraded. The Kolmogorov axioms require the probabilities of this exhaustive list of mutually exclusive outcomes to sum to 1.

Financial quantities such as returns, prices, and interest rates are often approximated with continuous random variables, even though observed market values may lie on discrete ticks and empirical distributions may contain atoms. A continuous distribution assigns probability zero to each individual point; this property does not require every intermediate real value to belong to its support. When the distribution is absolutely continuous, a probability density function gives interval probabilities by integration.

The PDF represents a fundamentally different concept than the PMF. Rather than giving the probability of exact values (which is zero for continuous variables), the PDF describes how probability is spread across the continuum of possible values. Higher density means probability is more concentrated in that region, while lower density means probability is more dispersed. The integral formula captures this: to find the probability of landing in an interval, we accumulate (integrate) the density across that interval.

The density itself is not a probability; it can exceed 1 for concentrated distributions. Only when we integrate over an interval do we obtain a probability between 0 and 1. Think of density as probability per unit length: a density of 2 at a point does not mean probability 2.

Both discrete and continuous random variables have cumulative distribution functions, giving a unified way to describe probability distributions. The CDF answers the running question "what's the probability of being less than or equal to this value?" as we move along the number line.

The CDF provides a universal framework for discrete and continuous random variables. It is nondecreasing and right-continuous, with limits 0 as $x \to -\infty$ and 1 as $x \to +\infty$. These properties make it useful for computing interval probabilities: for $a\leq b$, $P(a < X \leq b) = F_X(b) - F_X(a)$.

The CDF starts near 0 for very negative values and approaches 1 for large values. If the distribution is absolutely continuous, $F_X(x)=\int_{-\infty}^{x}f_X(u)\,du$ and $f_X=F_X'$ almost everywhere. A continuous CDF need not have a density, so this derivative relationship requires the stronger absolute-continuity condition.