Conditional Random Fields › Conditional Random Fields

Hidden Markov Models brought probabilistic reasoning to sequence labeling, but their generative formulation imposes strong constraints. Each observation depends only on its corresponding hidden state, forcing independence assumptions that conflict with the overlapping, correlated features that make natural language rich and complex. When you see the word "bank," its context matters enormously: preceding words, suffixes, capitalization patterns, surrounding punctuation. HMMs struggle to incorporate such features without violating their independence assumptions.

Conditional Random Fields (CRFs) emerged as a direct response to these limitations. By modeling the conditional probability of labels given observations rather than their joint distribution, CRFs can incorporate arbitrary overlapping features without worrying about dependencies between them. This discriminative approach has become the foundation of modern sequence labeling, powering named entity recognition, POS tagging, and countless other NLP tasks.

This chapter develops CRFs from first principles. We'll examine why HMMs fall short, derive the CRF formulation mathematically, implement feature functions, and build working CRF taggers. By the end, you'll understand how to use CRFs and why they work so well for sequence labeling.