BIO Tagging › Extended Tagging Schemes
Language AI HandbookAll chapters BIO Tagging › Extended Tagging SchemesOne section · Sections in this chapter3 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The basic BIO scheme is sufficient for many applications, but more complex annotation scenarios have motivated several extensions. These variants add prefixes to capture additional boundary information or handle special cases.
The four main schemes range from the simplest possible (IO) to the most expressive (BIOES):
| Scheme | Tags | Key Property |
|---|---|---|
| IO | I, O | Simplest but cannot distinguish adjacent same-type entities |
| BIO | B, I, O | Standard, handles all flat entity annotations |
| BIOES | B, I, O, E, S | Explicit end and single-token markers |
| BMEWO | B, M, E, W, O | Equivalent to BIOES with different naming |
The IO scheme is sometimes called the "naïve" scheme. It assigns I to every token inside any entity and O to everything else. You might wonder why this was ever used: as we saw, it cannot handle adjacent entities of the same type. Its only advantage is simplicity, having two labels instead of three or more. For tasks where the training data happens to have no adjacent same-type entities, IO performs identically to BIO. But this is a fragile assumption, and BIO's overhead of one additional label is so small that there is rarely a reason to choose IO.
At the other extreme, BIOES and its synonymous BILOU form offer richer supervision. The rationale is that teaching the model to recognize both starts and ends of entities, rather than just starts, helps it learn more precise boundary detection. Experiments on standard NER benchmarks have shown that models trained with BIOES sometimes outperform BIO-trained models by a small margin, particularly for longer entities and in settings where exact boundary matching is critical. The improvement is not universal, and the increased label vocabulary means more parameters in the output layer and more possible confusion between label types.
The BMEWO scheme renames the tags: B (beginning), M (middle, analogous to I for non-first non-last), E (end), W (whole, analogous to S for single-token), and O (outside). The semantics are identical to BIOES, and the choice between them is purely a matter of convention in different research communities or software libraries.
BIOES and BILOU
The BIOES scheme adds two more prefixes: E for the end of a multi-token entity and S for single-token entities. Some practitioners call this BILOU, using L (last) instead of E and U (unit) instead of S. The semantics are identical, only the names differ.
# BIOES tag meanings
bioes_tags = {
"B": "Beginning of multi-token entity",
"I": "Inside multi-token entity (not first, not last)",
"O": "Outside any entity",
"E": "End (last token) of multi-token entity",
"S": "Single-token entity (complete span)",
}
# Examples showing how BIOES differs from BIO
examples = [
(
["New", "York", "is", "great"],
["B-LOC", "E-LOC", "O", "O"],
"Two-token entity",
),
(["Paris", "is", "beautiful"], ["S-LOC", "O", "O"], "Single-token entity"),
(
["The", "United", "States", "of", "America"],
["O", "B-LOC", "I-LOC", "I-LOC", "E-LOC"],
"Multi-token entity",
),
]BIOES Tag Meanings: -------------------------------------------------- B: Beginning of multi-token entity I: Inside multi-token entity (not first, not last) O: Outside any entity E: End (last token) of multi-token entity S: Single-token entity (complete span) BIOES Examples: -------------------------------------------------- Two-token entity: New -> B-LOC York -> E-LOC is -> O great -> O Single-token entity: Paris -> S-LOC is -> O beautiful -> O Multi-token entity: The -> O United -> B-LOC States -> I-LOC of -> I-LOC America -> E-LOC
Why add more tags? BIOES provides two benefits. First, the model learns to recognize entity endpoints explicitly rather than inferring them from tag transitions. Research has shown modest accuracy improvements from BIOES over BIO, particularly for longer entities where boundary precision matters. Second, BIOES makes certain decoding errors impossible: a valid BIOES sequence must have every B eventually paired with an E, and S must stand alone without following B or I tags. These structural constraints can be enforced during decoding.
The S prefix is especially useful for single-token entities, which are common in NER. Names like "Paris" or "IBM" are frequent entities that don't need any B/I distinction. With BIO, a single-token entity uses B, which might be confusing since there is no continuation. With BIOES, the S prefix explicitly signals "this is a complete entity in itself."
There is also a subtle training advantage to the S prefix. When a model predicts S-LOC for "Paris," it is making a confident assertion about the entity's complete extent in a single decision. With BIO, the model predicts B-LOC for "Paris" and then has to predict O for the next token to implicitly close the entity. The BIO model therefore needs to learn a two-step decision for single-token entities, while BIOES encodes the same information in one step. In practice, the difference is small but consistent: BIOES models tend to have slightly better precision on single-token entities.
Valid Tag Transitions
Understanding which tag sequences are valid helps when designing decoders or training models with constraints. Not all tag combinations make sense. An I-PER cannot follow a B-LOC, and an I tag cannot appear after O without a preceding B tag.
import numpy as np
# Tags we will track (limited to 2 entity types for clarity)
tags_for_transition = ["O", "B-PER", "I-PER", "B-LOC", "I-LOC"]
n = len(tags_for_transition)
# Build transition validity matrix
# transitions[i, j] = 1 if tag[i] can be followed by tag[j]
transitions = np.zeros((n, n))
# O can be followed by: O, B-PER, B-LOC (not I tags)
transitions[0, :] = [1, 1, 0, 1, 0]
# B-PER can be followed by: O, B-PER, I-PER (same type), B-LOC
transitions[1, :] = [1, 1, 1, 1, 0]
# I-PER can be followed by: O, B-PER, I-PER (same type), B-LOC
transitions[2, :] = [1, 1, 1, 1, 0]
# B-LOC can be followed by: O, B-PER, B-LOC, I-LOC (same type)
transitions[3, :] = [1, 1, 0, 1, 1]
# I-LOC can be followed by: O, B-PER, B-LOC, I-LOC (same type)
transitions[4, :] = [1, 1, 0, 1, 1]
The key constraint is that I tags must match the type of their preceding B or I tag. An I-PER can follow B-PER or I-PER, but not B-LOC or I-LOC. This constraint can be enforced during inference using constrained beam search or CRF layers, improving the coherence of predicted tag sequences. We will explore CRF layers, which explicitly model these transitions as learned parameters, in the Conditional Random Fields chapter.
The transition constraint also explains why CRF decoders are particularly natural for BIO-tagged sequences. A neural model that predicts tag logits independently for each position might assign high probability to I-PER at position 5 even though the previous tag was B-LOC. The model's local context window might not catch this inconsistency. A CRF layer added on top of the neural logits explicitly scores entire tag sequences, penalizing forbidden transitions like I-PER after B-LOC. The Viterbi algorithm then finds the highest-scoring valid sequence. This global decoding step, which considers the entire tag sequence simultaneously, is precisely what makes CRF-augmented NER models so effective at maintaining entity boundary consistency.
The transition matrix also reveals a useful asymmetry: B tags and O have more liberal "next" options than I tags. You can transition from O to any B tag or to another O, but never to an I tag. B tags can transition to O, another B tag, or the same-type I tag. This asymmetry reflects the linguistic reality that entities can start anywhere but continuations are tightly constrained by what came before.