Hidden Markov Models (1970s) › A Concrete Example: Weather Prediction
History of Language AIAll chapters Hidden Markov Models (1970s) › A Concrete Example: Weather PredictionOne section · Sections in this chapter4 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
To make these abstract concepts concrete, consider a simplified HMM for weather prediction. Imagine you're locked in a windowless room and want to know about the weather outside. Your only information comes from observing what people are wearing when they enter the room. The actual weather conditions, whether it's sunny, cloudy, or rainy, are hidden from you. The clothing choices, shorts, light jackets, or heavy coats, are your only observable evidence.
In this scenario, the hidden states represent the actual weather conditions: sunny, cloudy, and rainy. These are the true states of the world that you cannot directly perceive. The observations are the clothing items you can see: shorts, light jackets, and heavy coats. Your goal is to infer the hidden weather from the observable clothing patterns.
The model captures two types of probabilistic relationships. Transition probabilities describe how weather conditions change over time. If it's sunny today, there's a high probability (0.8) it will remain sunny tomorrow, but also a moderate chance (0.2) it might become cloudy. Notice that you cannot jump directly from sunny to rainy in this model, you must pass through cloudy first. This reflects realistic weather patterns where conditions tend to change gradually rather than abruptly. From cloudy weather, you might transition to sunny (0.3), stay cloudy (0.5), or become rainy (0.2). Once rainy, there's a strong tendency to remain rainy (0.6) or clear up to cloudy (0.4).
Emission probabilities describe how likely each clothing choice is given the actual weather. When it's sunny, people heavily favor shorts (0.7) over light jackets (0.3), and nobody wears heavy coats. Cloudy weather leads to more light jackets (0.6) and some heavy coats (0.4), while rainy weather strongly favors heavy coats (0.8) though some people still opt for just light jackets (0.2).
With this model, even a single observation provides information. If you see someone wearing shorts, you can infer it's very likely sunny, because sunny weather has a high emission probability for shorts. But the real power emerges when you observe sequences. If you see shorts followed by light jackets followed by heavy coats over three consecutive days, the HMM can infer the most likely weather sequence was probably sunny, then cloudy, then rainy, even though you never directly observed the weather itself.
Here's an interactive visualization of this weather HMM:
The interactive graph above visualizes the complete structure of our weather HMM, showing the connections between hidden states and observations with probabilities labeled on each edge. Let's examine each component to understand how the pieces work together.
The blue circles represent the three hidden weather states defined above. In the HMM framework, these are the true underlying states that generate everything we can measure. They exist in the real world but remain hidden from direct observation.
The green circles represent observable states, the clothing people choose to wear: Shorts, Light Jacket, and Heavy Coat. These observations are our only window into the hidden weather conditions. Unlike the hidden states, we can directly measure and count clothing choices, making them our empirical data.
The gray arrows show transition edges, capturing how weather conditions evolve over time. Each arrow represents a possible state change with an associated probability. The thick arrow from Sunny to Sunny with probability 0.8 indicates a strong tendency for sunny weather to persist. Weather systems have momentum; they don't change randomly from moment to moment. The transitions encode this temporal structure. Notice that some transitions don't exist in this model. You cannot go directly from Sunny to Rainy; you must pass through Cloudy. This reflects domain knowledge about how weather patterns typically evolve.
The red arrows represent emission edges, connecting hidden states to the observations they generate. When it's Sunny, the model produces Shorts with probability 0.7 and Light Jacket with probability 0.3. These emission probabilities capture the relationship between the hidden weather and observable clothing choices. The probabilities reflect both the weather's influence on clothing choices and the natural variability in human behavior. Not everyone responds to the same weather in the same way, and emission probabilities model that variability.
The Power of Probabilistic Inference
The true power of this framework becomes apparent when you observe sequences over time. Suppose you see someone wearing a Heavy Coat three days in a row. Using the Viterbi algorithm, the HMM would decode this observation sequence and conclude that the weather was most likely Rainy on all three days, even though you never directly observed the weather. But the model doesn't just give you a single answer. It can compute the full probability distribution over possible weather sequences, quantifying your uncertainty.
This simple example captures the essential principle behind HMMs: using observable evidence to make principled, probabilistic inferences about hidden states that we cannot directly measure. The same mathematical framework that infers weather from clothing can infer phonemes from acoustic features, parts of speech from word sequences, or gene structures from DNA sequences. The abstraction is powerful precisely because it separates the general inference machinery from the specific domain details.