CRF Training › Training a Complete NER Model
Language AI HandbookAll chapters CRF Training › Training a Complete NER ModelOne section · Sections in this chapter8 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's put everything together and train a CRF on a real NER dataset. This section is both a demonstration of the full pipeline and a worked example of how the concepts from earlier sections appear in a practical setting. the training loop implicitly performs the forward-backward pass for each sentence during each L-BFGS iteration, how the regularization parameters shape the learned feature weights, and how the transition matrix encodes the label structure of the NER task.
from nltk.corpus import conll2002
# Load Spanish NER data (smaller than English, good for demonstration)
train_sents = list(conll2002.iob_sents("esp.train"))[
:100
] # Use subset for speed
test_sents = list(conll2002.iob_sents("esp.testa"))[:30]
def word2features(sent, i):
"""Extract features for word at position i in sentence."""
word = sent[i][0]
postag = sent[i][1]
features = {
"bias": 1.0,
"word.lower": word.lower(),
"word[-3:]": word[-3:],
"word[-2:]": word[-2:],
"word.isupper": word.isupper(),
"word.istitle": word.istitle(),
"word.isdigit": word.isdigit(),
"postag": postag,
"postag[:2]": postag[:2],
}
if i > 0:
word1 = sent[i - 1][0]
postag1 = sent[i - 1][1]
features.update(
{
"-1:word.lower": word1.lower(),
"-1:word.istitle": word1.istitle(),
"-1:postag": postag1,
}
)
else:
features["BOS"] = True
if i < len(sent) - 1:
word1 = sent[i + 1][0]
postag1 = sent[i + 1][1]
features.update(
{
"+1:word.lower": word1.lower(),
"+1:word.istitle": word1.istitle(),
"+1:postag": postag1,
}
)
else:
features["EOS"] = True
return features
def sent2features(sent):
return [word2features(sent, i) for i in range(len(sent))]
def sent2labels(sent):
return [label for token, postag, label in sent]
# Prepare data
X_train_ner = [sent2features(s) for s in train_sents]
y_train_ner = [sent2labels(s) for s in train_sents]
X_test_ner = [sent2features(s) for s in test_sents]
y_test_ner = [sent2labels(s) for s in test_sents]# Train the CRF
crf_ner = sklearn_crfsuite.CRF(
algorithm="lbfgs",
c1=0.1,
c2=0.1,
max_iterations=50,
all_possible_transitions=True,
)
crf_ner.fit(X_train_ner, y_train_ner)
# Make predictions
y_pred_ner = crf_ner.predict(X_test_ner)NER Model Training Results
============================================================
Training sentences: 100
Test sentences: 30
Labels: ['B-LOC', 'B-MISC', 'B-ORG', 'B-PER', 'I-LOC', 'I-MISC', 'I-ORG', 'I-PER', 'O']
Classification Report (entity labels only):
------------------------------------------------------------
precision recall f1-score support
B-LOC 0.750 0.484 0.588 31
B-ORG 0.643 0.250 0.360 36
B-PER 0.435 0.625 0.513 16
I-PER 0.452 0.560 0.500 25
B-MISC 0.200 0.375 0.261 8
I-ORG 0.421 0.333 0.372 24
I-LOC 0.000 0.000 0.000 19
I-MISC 0.317 0.722 0.441 18
micro avg 0.442 0.407 0.424 177
macro avg 0.402 0.419 0.379 177
weighted avg 0.464 0.407 0.400 177
Weighted F1 Score: 0.4003Example Predictions: ============================================================ Sentence 1: -------------------------------------------------- Token True Predicted Match -------------------------------------------------- Sao B-LOC B-PER ✗ Paulo I-LOC I-PER ✗ ( O O ✓ Brasil B-LOC B-LOC ✓ ) O O ✓ , O O ✓ 23 O O ✓ may O O ✓ ( O O ✓ EFECOM B-ORG B-ORG ✓ ... Sentence 2: -------------------------------------------------- Token True Predicted Match -------------------------------------------------- - O O ✓ Sentence 3: -------------------------------------------------- Token True Predicted Match -------------------------------------------------- La O O ✓ multinacional O O ✓ española O O ✓ Telefónica B-ORG O ✗ ha O O ✓ impuesto O O ✓ un O O ✓ récord O O ✓ mundial O O ✓ al O O ✓ ...
The trained CRF achieves reasonable performance even on this limited training set. The model correctly identifies many entities by learning patterns from the feature templates. Errors often occur at entity boundaries or with rare entity types that don't have enough training examples.
Even with just 100 training sentences, the model captures meaningful patterns. This is one of the strengths of CRFs relative to purely neural approaches: by encoding domain knowledge into features, you can get useful performance with limited data. A neural model trained on 100 sentences would struggle to generalize, while the CRF with good features can use general linguistic knowledge (capitalization, word shape, context) that applies across sentences. The tradeoff is that the CRF cannot discover patterns that you did not think to encode as features, while a neural model might discover them from raw text given enough data.
The transition matrix reveals that the CRF has learned BIO constraints. High weights appear for valid transitions like O→B-LOC (starting a location entity) and B-LOC→I-LOC (continuing a location). Low or negative weights appear for invalid transitions like O→I-LOC (continuation without a beginning).
This automatic discovery of BIO constraints is one of the most useful practical advantages of CRFs over independent classifiers. A per-token classifier trained to predict NER labels will sometimes predict I-PER at the start of a sentence, which violates the BIO convention and is meaningless. The CRF cannot make that mistake: the learned transition weight from O (or from the start symbol) to I-PER will be strongly negative, because the training data never contains such a transition. The model is implicitly learning the structural rules of the tagging scheme from examples, without needing those rules to be hard-coded.
An important nuance: the CRF only learns BIO constraints as soft preferences, not hard rules. If the evidence from the emission features is strong enough (e.g., the word strongly looks like it continues an entity), the model can still produce invalid BIO sequences when the transition cost does not overwhelm the emission score. Some practitioners handle this by using a post-processing step that converts CRF output to valid BIO sequences, or by using a constrained Viterbi decoding that enforces hard BIO constraints during inference.