BIO Tagging › Decoding BIO Tags to Spans
Language AI HandbookAll chapters BIO Tagging › Decoding BIO Tags to SpansOne section · Sections in this chapter5 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The inverse operation extracts entity spans from a sequence of BIO tags. This is essential for evaluating model predictions and converting output to a usable format. The decoder maintains state across tokens, tracking whether we are inside an entity and of what type.
def bio_to_spans(tokens, tags):
"""
Extract entity spans from BIO-tagged sequence.
Handles malformed sequences gracefully:
- Orphan I tags treated as beginning a new entity
- Type mismatches close old entity and start new one
Returns list of (start_idx, end_idx, entity_type, text) tuples.
"""
spans = []
current_entity = None # Tracks (start_idx, entity_type)
for i, (token, tag) in enumerate(zip(tokens, tags)):
if tag.startswith("B-"):
# Close any open entity, then start a new one
if current_entity is not None:
start, etype = current_entity
spans.append((start, i, etype))
entity_type = tag[2:]
current_entity = (i, entity_type)
elif tag.startswith("I-"):
entity_type = tag[2:]
if current_entity is None:
# Orphan I: treat as a beginning
current_entity = (i, entity_type)
elif current_entity[1] != entity_type:
# Type mismatch: close old entity, start new one
start, etype = current_entity
spans.append((start, i, etype))
current_entity = (i, entity_type)
# Otherwise: continue current entity (do nothing)
else: # O tag
if current_entity is not None:
start, etype = current_entity
spans.append((start, i, etype))
current_entity = None
# Handle entity extending to end of sequence
if current_entity is not None:
start, etype = current_entity
spans.append((start, len(tokens), etype))
# Attach text for each span
return [(s, e, t, " ".join(tokens[s:e])) for s, e, t in spans]
# Test decoding
test_decode_tokens = ["Barack", "Obama", "visited", "New", "York", "City"]
test_decode_tags = ["B-PER", "I-PER", "O", "B-LOC", "I-LOC", "I-LOC"]
decoded_spans = bio_to_spans(test_decode_tokens, test_decode_tags)BIO to Span Decoding: -------------------------------------------------- Input tokens: Barack Obama visited New York City Input tags: B-PER I-PER O B-LOC I-LOC I-LOC Extracted entities: [0:2] 'Barack Obama' -> PER [3:6] 'New York City' -> LOC
The decoder correctly reconstructs both entities. Notice how the state machine approach handles the end of the sequence: if we reach the final token while inside an entity, the entity is closed at position len(tokens).
The state machine framing is worth dwelling on. At any position in the sequence, the decoder is in one of two states: either inside an entity of a specific type, or outside all entities. The current tag determines what transition to make. An O tag always moves to the "outside" state. A B tag always starts a new entity, possibly closing the current one first. An I tag either extends the current entity (if the types match) or handles the error case. This is a classic pattern in sequence processing: keeping just enough state to make local decisions while deferring final output until you know an entity is complete.
The design choice to return the entity text directly, computed by joining the token slice, is a practical convenience. In a real pipeline, you would want to return both the span indices (for indexing into the original text) and the reconstructed string (for display or lookup). The joined string is correct for whitespace-tokenized text but may not reconstruct the original surface form perfectly for tokenizers that handle punctuation differently.
Handling Malformed Sequences
Model predictions don't always produce valid BIO sequences. Common errors include I tags without a preceding B tag and type mismatches where I-LOC follows B-PER. A defensive decoder must handle these gracefully:
# Three common malformed sequence patterns
malformed_cases = [
{
"name": "Orphan I tag (no preceding B)",
"tokens": ["went", "to", "York", "City"],
"tags": ["O", "O", "I-LOC", "I-LOC"],
},
{
"name": "Type mismatch in continuation",
"tokens": ["John", "Smith", "Jr"],
"tags": ["B-PER", "I-PER", "I-ORG"], # I-ORG after PER entity
},
{
"name": "Consecutive B tags (no I between them)",
"tokens": ["Paris", "London", "Berlin"],
"tags": ["B-LOC", "B-LOC", "B-LOC"],
},
]
malformed_results = []
for case in malformed_cases:
spans = bio_to_spans(case["tokens"], case["tags"])
malformed_results.append(
{
"name": case["name"],
"tokens": case["tokens"],
"tags": case["tags"],
"spans": spans,
}
)Handling Malformed Sequences:
============================================================
Orphan I tag (no preceding B):
Tokens: ['went', 'to', 'York', 'City']
Tags: ['O', 'O', 'I-LOC', 'I-LOC']
Decoded entities:
[2:4] 'York City' -> LOC
Type mismatch in continuation:
Tokens: ['John', 'Smith', 'Jr']
Tags: ['B-PER', 'I-PER', 'I-ORG']
Decoded entities:
[0:2] 'John Smith' -> PER
[2:3] 'Jr' -> ORG
Consecutive B tags (no I between them):
Tokens: ['Paris', 'London', 'Berlin']
Tags: ['B-LOC', 'B-LOC', 'B-LOC']
Decoded entities:
[0:1] 'Paris' -> LOC
[1:2] 'London' -> LOC
[2:3] 'Berlin' -> LOCThe decoder applies sensible recovery strategies. Orphan I tags are treated as beginning a new entity. Type mismatches close the previous entity and start a fresh one. Consecutive B tags produce separate single-token entities. These choices maximize recall at the cost of some precision, which is often preferable for downstream error analysis.
The philosophy behind these recovery strategies deserves some explanation. When evaluating a model, you want to compare its predictions against gold annotations as fairly as possible. A decoder that rejects malformed sequences outright would throw away potentially useful partial information: a model that predicted I-LOC instead of B-LOC at the right position is closer to correct than a model that predicted O. The lenient decoder recovers what it can, allowing evaluation metrics to distinguish degrees of error rather than treating all invalid sequences identically.
In a production NER system, you might choose stricter behavior: log all malformed sequences for human review rather than silently recovering them. This is particularly important during dataset preparation, where silent repairs can mask systematic annotation errors in your training data. A checker that reports every malformed sequence during annotation helps annotators maintain consistent boundary conventions.
BIOES Decoding
Decoding BIOES follows the same state machine pattern, with additional transitions for the S and E prefixes:
def bioes_to_spans(tokens, tags):
"""
Extract entity spans from BIOES-tagged sequence.
S tags produce single-token spans directly.
E tags close the current entity at the current position.
"""
spans = []
current_entity = None
for i, (token, tag) in enumerate(zip(tokens, tags)):
if tag.startswith("S-"):
# Close any open entity, then add this single-token entity
if current_entity is not None:
start, etype = current_entity
spans.append((start, i, etype))
current_entity = None
entity_type = tag[2:]
spans.append((i, i + 1, entity_type))
elif tag.startswith("B-"):
if current_entity is not None:
start, etype = current_entity
spans.append((start, i, etype))
entity_type = tag[2:]
current_entity = (i, entity_type)
elif tag.startswith("I-"):
entity_type = tag[2:]
if current_entity is None or current_entity[1] != entity_type:
if current_entity is not None:
start, etype = current_entity
spans.append((start, i, etype))
current_entity = (i, entity_type)
elif tag.startswith("E-"):
# Close entity at current position (inclusive)
entity_type = tag[2:]
if current_entity is not None and current_entity[1] == entity_type:
start, _ = current_entity
spans.append((start, i + 1, entity_type))
else:
# Orphan E: treat as single-token entity
spans.append((i, i + 1, entity_type))
current_entity = None
else: # O tag
if current_entity is not None:
start, etype = current_entity
spans.append((start, i, etype))
current_entity = None
if current_entity is not None:
start, etype = current_entity
spans.append((start, len(tokens), etype))
return [(s, e, t, " ".join(tokens[s:e])) for s, e, t in spans]
# Test BIOES decoding
bioes_tokens = ["John", "visited", "New", "York", "and", "Paris"]
bioes_tags = ["S-PER", "O", "B-LOC", "E-LOC", "O", "S-LOC"]
bioes_decoded = bioes_to_spans(bioes_tokens, bioes_tags)BIOES Decoding: -------------------------------------------------- Input: John visited New York and Paris Tags: S-PER O B-LOC E-LOC O S-LOC Extracted entities: [0:1] 'John' -> PER [2:4] 'New York' -> LOC [5:6] 'Paris' -> LOC
The S tags directly produce single-token entities, while B-E pairs define multi-token spans. This explicit boundary marking simplifies validation: you can check that every B is eventually followed by a matching E, and that S tags stand alone.