Conditional Random Fields (2001) › The Role of Feature Functions
History of Language AIAll chapters Conditional Random Fields (2001) › The Role of Feature FunctionsOne section · Sections in this chapter4 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The true power of CRFs emerges from how they use feature functions to capture different types of regularities in language. These functions serve as the building blocks through which the model understands the relationships between words and their labels. Researchers working with CRFs typically design several categories of features, each capturing a different aspect of the structured prediction problem.
State features examine how well a particular label fits a specific word given its characteristics. For instance, a state feature might fire when the word is "cat" and the proposed label is "noun," encoding the intuition that this word commonly functions as a noun in English. These features can consider various properties of the word itself, such as its spelling, capitalization, suffixes, or position in the sentence. State features essentially answer the question, "Does this label make sense for this particular word in isolation?"
Transition features capture patterns in how labels tend to follow one another. A transition feature might fire when a determiner label is followed by a noun label, reflecting the grammatical regularity that determiners typically precede nouns in English. These features encode the sequential dependencies that make CRFs effective for structured prediction. They answer the question, "Given that we assigned this label to the previous word, how plausible is this label for the current word?"
Beyond these two fundamental categories, researchers often design context features that look at a broader window of surrounding words to inform decisions. A context feature might consider whether the word two positions ahead looks like a verb when deciding how to label the current word. Some systems even incorporate global features that capture properties of the entire sentence that might influence local labeling decisions.
The beauty of this feature-based approach is its flexibility. Researchers can encode linguistic knowledge, intuitions from examining data, and domain-specific insights directly into the feature set. The CRF learning algorithm then determines which features are truly predictive and how much weight each should receive in making predictions.