BIO Tagging › Multi-Label BIO Tagging
Language AI HandbookAll chapters BIO Tagging › Multi-Label BIO TaggingOne section · Sections in this chapter7 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Standard BIO tagging assumes each token belongs to at most one entity. But some applications require overlapping annotations. Consider "Bank of America," which might be tagged as both an organization (the company) and a location ("America" is a place). Nested named entities present similar challenges.
The need for multi-label or nested annotations arises frequently in specialized domains. Biomedical text is the canonical example: a phrase like "EGFR inhibitor resistance" might simultaneously span a gene ("EGFR"), a chemical class ("inhibitor"), and a medical concept ("EGFR inhibitor resistance") as a clinical finding. The ACE 2004 and 2005 annotation corpora, used in relation extraction research, include nested entities where a person mention can contain an organization name and vice versa. Scientific literature often contains nested mentions where a model name or acronym is embedded within a longer description.
Several approaches handle multi-label scenarios:
# Approach 1: Multiple tag columns (one per entity type)
multi_column_example = {
"tokens": ["Bank", "of", "America", "CEO"],
"ORG_tags": ["B-ORG", "I-ORG", "I-ORG", "O"],
"LOC_tags": ["O", "O", "B-LOC", "O"],
}
# Approach 2: Separate passes per entity type
separate_passes = {
"pass_1_ORG": ["B-ORG", "I-ORG", "I-ORG", "O"],
"pass_2_LOC": ["O", "O", "B-LOC", "O"],
}
# Approach 3: Combined labels (for small, fixed overlapping sets)
combined_example = {
"tokens": ["Bank", "of", "America", "CEO"],
"combined": ["B-ORG", "I-ORG", "I-ORG+B-LOC", "O"],
}Multi-Label BIO: Three Approaches ======================================================= 1. Multiple Tag Columns (one per entity type): Token ORG LOC ----------------------------------- Bank B-ORG O of I-ORG O America I-ORG B-LOC CEO O O 2. Combined Labels (for overlapping spans): Token Combined ----------------------------------- Bank B-ORG of I-ORG America I-ORG+B-LOC CEO O
The multiple-column approach is cleanest but requires training separate models or a model with multiple output heads. Combined tags work for small label sets but explode combinatorially with many types. In practice, most NER systems use flat BIO tagging and handle overlaps through post-processing or by defining a type hierarchy.
The multiple-output-head approach is the current best practice for multi-type NER. Modern transformer-based NER models like those built on BERT can easily support multiple output heads: one shared encoder produces contextual token representations, and separate linear layers on top map those representations to tag logits for each entity type. The loss is computed independently for each head and summed. This architecture scales cleanly to any number of entity types and handles complete overlap (a token tagged as both PER and ORG by different heads) without any special combinatorial logic.
Nested Entity Encoding
For nested entities like "New York University" where "New York" is LOC and "New York University" is ORG, specialized schemes use layered BIO tags, one layer per nesting depth:
def encode_nested_entities(tokens, entities):
"""
Encode nested entities using layered BIO tags.
Larger (outer) spans get assigned to earlier layers.
Smaller (inner) spans fill later layers.
"""
# Determine how many layers we need
# A token can appear in at most as many layers as entities cover it
max_depth = 0
for pos in range(len(tokens)):
depth = sum(1 for s, e, _ in entities if s <= pos < e)
max_depth = max(max_depth, depth)
if max_depth == 0:
return [["O"] * len(tokens)]
layers = [["O"] * len(tokens) for _ in range(max_depth)]
# Assign larger spans first so outer entities go to earlier layers
sorted_entities = sorted(entities, key=lambda x: (-(x[1] - x[0]), x[0]))
for start, end, etype in sorted_entities:
# Find the first layer where this span is fully available
for layer in layers:
if all(layer[i] == "O" for i in range(start, end)):
layer[start] = f"B-{etype}"
for i in range(start + 1, end):
layer[i] = f"I-{etype}"
break
return layers
# Nested entity example
nested_tokens = ["New", "York", "University", "is", "great"]
nested_entities = [
(0, 3, "ORG"), # New York University (outer)
(0, 2, "LOC"), # New York (inner)
]
nested_layers = encode_nested_entities(nested_tokens, nested_entities)Nested Entity Encoding: -------------------------------------------------- Token Layer 1 Layer 2 -------------------------------------------------- New B-ORG B-LOC York I-ORG I-LOC University I-ORG O is O O great O O Layer 1: outer ORG span (New York University) Layer 2: inner LOC span (New York)
This layered approach preserves all entity information but requires models that can predict multiple layers simultaneously. Modern nested NER systems often use span-based prediction instead, directly outputting all valid spans regardless of nesting depth.
The span-based approach treats NER as a classification problem over candidate spans rather than a sequence labeling problem over individual tokens. Every possible span (or every span up to some maximum length) is enumerated and fed through a classifier that decides whether it is an entity and, if so, what type. Span-based models handle nested entities naturally because different spans can be classified independently, without any constraint that each token belongs to exactly one entity. The tradeoff is computational cost: enumerating all spans is quadratic in sentence length, whereas sequence tagging is linear.