Conditional Random Fields (2001) › A Concrete Example
History of Language AIAll chapters Conditional Random Fields (2001) › A Concrete ExampleOne section · Sections in this chapter5 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
To see how these pieces fit together in practice, let us walk through a concrete example of part-of-speech tagging. Consider again the sentence "The cat sat on the mat." A CRF trained for this task would evaluate many possible label sequences and assign each a probability based on how well the features match the input.
For the correct labeling, DT NN VB IN DT NN, many state features would fire strongly. The model would note that "The" exhibits characteristics typical of determiners, being a common function word that appears at the start of phrases. The word "cat" has the spelling pattern and usage frequency of a noun. The word "sat" follows verb morphology with its past tense suffix. The word "on" appears in the model's learned list of prepositions. Each of these observations contributes evidence that the proposed labels fit the observed words.
Simultaneously, the transition features would provide strong support for this labeling. The sequence begins with a determiner followed by a noun, one of the most common patterns in English noun phrases. Then a noun is followed by a verb, reflecting the typical subject-verb structure of simple sentences. The verb is followed by a preposition, a common pattern when verbs take prepositional phrase complements. Finally, the preposition is followed by another determiner-noun sequence, the standard structure of a prepositional phrase.
The CRF combines all this evidence through the weighted sum in its probability formula. Features that strongly indicate the correctness of this labeling contribute positive scores. The exponential and normalization then convert these scores into a probability. Alternative labelings would receive lower scores because fewer features would fire, or features with negative weights would activate, pulling down the overall probability.
Through training on labeled examples, the model learns which features are reliable indicators and how much weight each should receive. Features that consistently help identify correct labelings gain positive weights, while features that fire for incorrect labelings acquire negative or near-zero weights. The result is a system that can balance multiple sources of evidence to find the most probable label sequence for any input sentence.