BIO Tagging for Named Entity Recognition

Sequence labeling tasks like named entity recognition face a fundamental challenge: how do you represent multi-word entities using per-token labels? The sentence "New York City is beautiful" contains three tokens that together form a single location entity. Assigning all three the label "LOC" creates ambiguity. Does "New" start a new entity, or does it continue one that began earlier? Are "New," "York," and "City" three separate locations, or one?

BIO tagging solves this problem elegantly. The scheme uses a small set of prefixes to encode entity boundaries directly in the labels. B marks the beginning of an entity, I marks inside (continuation), and O marks outside (no entity). With BIO tags, "New York City" becomes B-LOC I-LOC I-LOC, unambiguously marking a single three-token entity.

The idea is deceptively simple. Three letters carry enough information to reconstruct any flat entity annotation from scratch. Any system reading the tag sequence can tell exactly where each entity starts, where it continues, and where the text returns to non-entity territory, without any additional context or lookup table. That parsimony, combined with perfect expressiveness for non-overlapping spans, explains why BIO tagging became the standard format for nearly every NER dataset and evaluation benchmark produced in the past two decades.

This chapter explores BIO tagging from its basic mechanics through practical implementation. You will learn the standard BIO scheme and its variants, implement converters between span annotations and BIO tags, build decoders that extract entities from tagged sequences, and handle the edge cases that arise in real-world tagging scenarios. Building on the entity types from the Named Entity Recognition chapter, here we focus on the encoding mechanism that makes those entity types usable in sequence models. The concepts here apply directly to the Hidden Markov Model and Conditional Random Field chapters, where you will see how these tag sequences get used as training targets and how transition constraints between tags can be learned from data.

BIO tagging encodes entity boundaries through prefix annotations. Each token receives a label combining a position indicator (B, I, or O) with an entity type. The three positions work together to delimit entity spans without ambiguity.

The design is minimal but complete. You need exactly three pieces of information to reconstruct any flat (non-overlapping) entity annotation: where does the entity start, where does it continue, and where is there no entity. B, I, and O capture precisely these three cases. Any more prefixes would be redundant; any fewer would lose boundary information.

The scheme originated in work on text chunking and part-of-speech tagging before NER, where researchers needed a way to identify multi-word phrases using per-word labels. When NER became a major shared task at conferences like CoNLL in the early 2000s, BIO tagging was adopted as the standard label format because annotators, training pipelines, and evaluation scripts could all work with the same consistent representation. The CoNLL-2003 dataset, which remains a standard English NER benchmark, uses BIO tags over four entity types (PER, LOC, ORG, MISC) and set the template that most NER datasets have followed since.

Let's see how BIO tagging works on a concrete example:

The BIO scheme achieves two critical goals. First, it marks entity boundaries explicitly. When you see a B tag, you know a new entity starts at that position. When you see an I tag following a B tag of the same type, you know the entity continues. Second, it handles adjacent entities correctly. If "Barack Obama" and "Michelle Obama" appeared consecutively without a gap, the B prefix on "Michelle" would clearly mark the second entity's start, producing B-PER I-PER B-PER I-PER.

It is worth pausing to appreciate why this property matters so much in practice. Annotators frequently tag lists of proper nouns: "John Smith, Jane Doe, and Robert Johnson attended the summit." Without the B marker, a reading of consecutive PER tokens would be ambiguous about how many people are named. With BIO, each B-PER resets the entity counter, so even a list of ten names is perfectly unambiguous. The same logic applies to location lists in news text ("Paris, Berlin, and London"), organization lists in financial documents, and product names in e-commerce text.

A simpler approach might label each token with just its entity type: PER, LOC, or O. Let's see why this fails.

Without the B prefix, there is no way to determine where one entity ends and another begins. The simple scheme makes adjacent same-type entities indistinguishable from single multi-token entities. Real text contains many such cases: lists of names, multiple locations in a sentence, consecutive organization mentions. BIO tagging handles all of them correctly.

This failure mode is not theoretical. Consider sports reporting: "Federer Djokovic Nadal will all compete." Or financial news: "Microsoft Google Apple reported earnings." Labeling these with simple type tags produces one enormous entity rather than three distinct ones. Any downstream application that counts entities, links them to databases, or resolves coreferences would produce completely wrong results from the simpler scheme.

The deeper reason BIO works is that it encodes both the type of each token and its structural role within an entity. The structural information, whether a token is the start, the interior, or entirely outside, is independent of the type information and adds a structural dimension that flat type labels cannot provide.

The O tag marks tokens that don't belong to any entity. It carries no suffix because "outside" is the only interpretation, there is no entity type to specify. In typical NER datasets, O tokens vastly outnumber entity tokens since most words in a sentence are not named entities:

This class imbalance, where O tokens dominate, is characteristic of sequence labeling tasks. Training algorithms must account for it, often through weighted loss functions or sampling strategies. A baseline model that always predicts O achieves deceptively high accuracy but zero utility, since it never identifies any entity.

The imbalance is especially severe for fine-grained entity types. In a dataset with 18 entity types, each individual type might account for only one or two percent of all tokens. If you train a model that treats every token prediction independently with equal loss weights, the model learns that the safest policy is to predict O for everything, since the loss from missing an entity is small compared to the aggregate loss from misclassifying the overwhelming majority of non-entity tokens. Weighted cross-entropy, focal loss, and oversampling entity-bearing sentences all address this, but the fundamental imbalance is a structural property of language: most words in any document are connective tissue rather than named referents.

The basic BIO scheme is sufficient for many applications, but more complex annotation scenarios have motivated several extensions. These variants add prefixes to capture additional boundary information or handle special cases.

The four main schemes range from the simplest possible (IO) to the most expressive (BIOES):

The IO scheme is sometimes called the "naïve" scheme. It assigns I to every token inside any entity and O to everything else. You might wonder why this was ever used: as we saw, it cannot handle adjacent entities of the same type. Its only advantage is simplicity, having two labels instead of three or more. For tasks where the training data happens to have no adjacent same-type entities, IO performs identically to BIO. But this is a fragile assumption, and BIO's overhead of one additional label is so small that there is rarely a reason to choose IO.

At the other extreme, BIOES and its synonymous BILOU form offer richer supervision. The rationale is that teaching the model to recognize both starts and ends of entities, rather than just starts, helps it learn more precise boundary detection. Experiments on standard NER benchmarks have shown that models trained with BIOES sometimes outperform BIO-trained models by a small margin, particularly for longer entities and in settings where exact boundary matching is critical. The improvement is not universal, and the increased label vocabulary means more parameters in the output layer and more possible confusion between label types.

The BMEWO scheme renames the tags: B (beginning), M (middle, analogous to I for non-first non-last), E (end), W (whole, analogous to S for single-token), and O (outside). The semantics are identical to BIOES, and the choice between them is purely a matter of convention in different research communities or software libraries.

The BIOES scheme adds two more prefixes: E for the end of a multi-token entity and S for single-token entities. Some practitioners call this BILOU, using L (last) instead of E and U (unit) instead of S. The semantics are identical, only the names differ.

Why add more tags? BIOES provides two benefits. First, the model learns to recognize entity endpoints explicitly rather than inferring them from tag transitions. Research has shown modest accuracy improvements from BIOES over BIO, particularly for longer entities where boundary precision matters. Second, BIOES makes certain decoding errors impossible: a valid BIOES sequence must have every B eventually paired with an E, and S must stand alone without following B or I tags. These structural constraints can be enforced during decoding.

The S prefix is especially useful for single-token entities, which are common in NER. Names like "Paris" or "IBM" are frequent entities that don't need any B/I distinction. With BIO, a single-token entity uses B, which might be confusing since there is no continuation. With BIOES, the S prefix explicitly signals "this is a complete entity in itself."

There is also a subtle training advantage to the S prefix. When a model predicts S-LOC for "Paris," it is making a confident assertion about the entity's complete extent in a single decision. With BIO, the model predicts B-LOC for "Paris" and then has to predict O for the next token to implicitly close the entity. The BIO model therefore needs to learn a two-step decision for single-token entities, while BIOES encodes the same information in one step. In practice, the difference is small but consistent: BIOES models tend to have slightly better precision on single-token entities.

Understanding which tag sequences are valid helps when designing decoders or training models with constraints. Not all tag combinations make sense. An I-PER cannot follow a B-LOC, and an I tag cannot appear after O without a preceding B tag.

The key constraint is that I tags must match the type of their preceding B or I tag. An I-PER can follow B-PER or I-PER, but not B-LOC or I-LOC. This constraint can be enforced during inference using constrained beam search or CRF layers, improving the coherence of predicted tag sequences. We will explore CRF layers, which explicitly model these transitions as learned parameters, in the Conditional Random Fields chapter.

The transition constraint also explains why CRF decoders are particularly natural for BIO-tagged sequences. A neural model that predicts tag logits independently for each position might assign high probability to I-PER at position 5 even though the previous tag was B-LOC. The model's local context window might not catch this inconsistency. A CRF layer added on top of the neural logits explicitly scores entire tag sequences, penalizing forbidden transitions like I-PER after B-LOC. The Viterbi algorithm then finds the highest-scoring valid sequence. This global decoding step, which considers the entire tag sequence simultaneously, is precisely what makes CRF-augmented NER models so effective at maintaining entity boundary consistency.

The transition matrix also reveals a useful asymmetry: B tags and O have more liberal "next" options than I tags. You can transition from O to any B tag or to another O, but never to an I tag. B tags can transition to O, another B tag, or the same-type I tag. This asymmetry reflects the linguistic reality that entities can start anywhere but continuations are tightly constrained by what came before.

Annotation tools often store entity information as character or token spans rather than per-token labels. Converting these span annotations to BIO format is a common preprocessing step. Let.s build a converter that handles common edge cases.

The conversion logic is straightforward: initialize all tokens as O, then for each span, mark the first token with B and all subsequent tokens with I.

Single-token entities receive only a B tag since there is no continuation token. Adjacent same-type entities each start with B, correctly distinguishing them. Entity spans at sequence boundaries are handled without special-casing since the index bounds check catches out-of-range spans.

The adjacent same-type case is particularly important to verify. The converter sorts spans by start position and processes them in order, so when "Obama" (span 0-1) and "Biden" (span 1-2) are both PER, the first loop iteration sets tags[0] = "B-PER", and the second sets tags[1] = "B-PER". The result is two separate single-token entities rather than a merged two-token entity, which is exactly correct. This deterministic behavior relies on the span representation distinguishing between (0, 1) and (1, 2), which is why exclusive end indices are the right convention.

For applications requiring BIOES format, we extend the converter to track entity boundaries and assign the appropriate suffix:

The BIOES output makes the entity endpoint explicit: "Times" receives E-ORG rather than I-ORG, marking it as the final token in the span. This is the key difference: with BIO, you can only determine that an entity has ended after seeing a B or O tag at the next position; with BIOES, the E tag tells you immediately.

This "look-ahead" property of BIO is a subtle but real limitation. During sequential decoding, a BIO decoder reading token by token does not know whether the current I tag is the last one in the entity until it reads the next token. This means the decoder cannot output complete entity spans until it sees what comes after. BIOES eliminates this look-ahead dependency: an E tag immediately signals "the entity I've been tracking ends here." For streaming applications or interactive systems that display entity annotations as they process each token, BIOES provides a cleaner interface.

The inverse operation extracts entity spans from a sequence of BIO tags. This is essential for evaluating model predictions and converting output to a usable format. The decoder maintains state across tokens, tracking whether we are inside an entity and of what type.

The decoder correctly reconstructs both entities. Notice how the state machine approach handles the end of the sequence: if we reach the final token while inside an entity, the entity is closed at position len(tokens).

The state machine framing is worth dwelling on. At any position in the sequence, the decoder is in one of two states: either inside an entity of a specific type, or outside all entities. The current tag determines what transition to make. An O tag always moves to the "outside" state. A B tag always starts a new entity, possibly closing the current one first. An I tag either extends the current entity (if the types match) or handles the error case. This is a classic pattern in sequence processing: keeping just enough state to make local decisions while deferring final output until you know an entity is complete.

The design choice to return the entity text directly, computed by joining the token slice, is a practical convenience. In a real pipeline, you would want to return both the span indices (for indexing into the original text) and the reconstructed string (for display or lookup). The joined string is correct for whitespace-tokenized text but may not reconstruct the original surface form perfectly for tokenizers that handle punctuation differently.

Model predictions don't always produce valid BIO sequences. Common errors include I tags without a preceding B tag and type mismatches where I-LOC follows B-PER. A defensive decoder must handle these gracefully:

The decoder applies sensible recovery strategies. Orphan I tags are treated as beginning a new entity. Type mismatches close the previous entity and start a fresh one. Consecutive B tags produce separate single-token entities. These choices maximize recall at the cost of some precision, which is often preferable for downstream error analysis.

The philosophy behind these recovery strategies deserves some explanation. When evaluating a model, you want to compare its predictions against gold annotations as fairly as possible. A decoder that rejects malformed sequences outright would throw away potentially useful partial information: a model that predicted I-LOC instead of B-LOC at the right position is closer to correct than a model that predicted O. The lenient decoder recovers what it can, allowing evaluation metrics to distinguish degrees of error rather than treating all invalid sequences identically.

In a production NER system, you might choose stricter behavior: log all malformed sequences for human review rather than silently recovering them. This is particularly important during dataset preparation, where silent repairs can mask systematic annotation errors in your training data. A checker that reports every malformed sequence during annotation helps annotators maintain consistent boundary conventions.

Decoding BIOES follows the same state machine pattern, with additional transitions for the S and E prefixes:

The S tags directly produce single-token entities, while B-E pairs define multi-token spans. This explicit boundary marking simplifies validation: you can check that every B is eventually followed by a matching E, and that S tags stand alone.

Real-world tagging systems produce inconsistent output. A well-designed pipeline includes validation to detect problems and, where possible, repair them automatically.

Validation matters at two distinct pipeline stages. Before training, you validate annotation data to catch inconsistencies introduced during the labeling process. Annotators may disagree on boundaries, tools may produce off-by-one errors, and automatic pre-annotation may generate malformed sequences that human annotators did not catch. Feeding invalid sequences to a sequence model during training can produce subtle bugs: the model might learn spurious patterns from incorrectly tagged tokens, or the loss computation might silently ignore malformed positions.

After inference, you validate model output to ensure it can be parsed correctly downstream. Even a model trained entirely on valid sequences may occasionally predict invalid ones, particularly on out-of-distribution inputs where the model is uncertain. The application reading NER output may crash or produce wrong results if it assumes valid sequences and receives invalid ones.

The repair visualization highlights positions that changed (shown with orange borders). Position 1 was an orphan I-PER that became B-PER. Position 4 was a type mismatch I-ORG that became B-ORG. Both repairs create clean entity boundaries that the decoder can now process correctly.

It is important to log all repairs rather than applying them silently. When you repair sequences during training data preparation, the log tells you whether your annotation pipeline has systematic problems. If 20% of sequences require repairs, something is wrong with your annotation tool or guidelines. If only 0.1% require repairs, you have an isolated edge case. The repair log is also a diagnostic tool during model development: when a model's output requires many repairs at inference time, it signals the model is confused about boundary detection and may need targeted fine-tuning on boundary examples.

Standard BIO tagging assumes each token belongs to at most one entity. But some applications require overlapping annotations. Consider "Bank of America," which might be tagged as both an organization (the company) and a location ("America" is a place). Nested named entities present similar challenges.

The need for multi-label or nested annotations arises frequently in specialized domains. Biomedical text is the canonical example: a phrase like "EGFR inhibitor resistance" might simultaneously span a gene ("EGFR"), a chemical class ("inhibitor"), and a medical concept ("EGFR inhibitor resistance") as a clinical finding. The ACE 2004 and 2005 annotation corpora, used in relation extraction research, include nested entities where a person mention can contain an organization name and vice versa. Scientific literature often contains nested mentions where a model name or acronym is embedded within a longer description.

Several approaches handle multi-label scenarios:

The multiple-column approach is cleanest but requires training separate models or a model with multiple output heads. Combined tags work for small label sets but explode combinatorially with many types. In practice, most NER systems use flat BIO tagging and handle overlaps through post-processing or by defining a type hierarchy.

The multiple-output-head approach is the current best practice for multi-type NER. Modern transformer-based NER models like those built on BERT can easily support multiple output heads: one shared encoder produces contextual token representations, and separate linear layers on top map those representations to tag logits for each entity type. The loss is computed independently for each head and summed. This architecture scales cleanly to any number of entity types and handles complete overlap (a token tagged as both PER and ORG by different heads) without any special combinatorial logic.

For nested entities like "New York University" where "New York" is LOC and "New York University" is ORG, specialized schemes use layered BIO tags, one layer per nesting depth:

This layered approach preserves all entity information but requires models that can predict multiple layers simultaneously. Modern nested NER systems often use span-based prediction instead, directly outputting all valid spans regardless of nesting depth.

The span-based approach treats NER as a classification problem over candidate spans rather than a sequence labeling problem over individual tokens. Every possible span (or every span up to some maximum length) is enumerated and fed through a classifier that decides whether it is an entity and, if so, what type. Span-based models handle nested entities naturally because different spans can be classified independently, without any constraint that each token belongs to exactly one entity. The tradeoff is computational cost: enumerating all spans is quadratic in sentence length, whereas sequence tagging is linear.

Measuring NER system accuracy requires comparing predicted BIO sequences against gold-standard annotations. The standard evaluation protocol counts entities at the span level, not the token level. A prediction is correct only if both the entity boundaries and the entity type match the gold standard exactly.

This strict matching criterion has practical effects. Consider a gold annotation of "New York City" as LOC. A model that predicts B-LOC I-LOC for "New York" (missing "City") scores zero on this entity. Neither a partial credit nor a type-only credit is awarded. The full-span requirement means that boundary errors are penalized just as heavily as type errors. This motivates the attention to boundary precision we saw in the BIOES scheme discussion.

The evaluation results illustrate the strictness of span-level matching. A boundary error that misses just one token of a three-token entity scores F1=0.5, the same as a type error where the entire entity is correctly bounded but labeled as the wrong type. Both errors are completely wrong from the perspective of exact span matching. A system designer choosing between BIOES and BIO might find this motivating: if boundary errors are equally costly as type errors, the explicit end markers of BIOES, which help the model learn exact boundaries, can directly improve F1.

Let's consolidate our functions into a reusable class and demonstrate a complete end-to-end workflow integrating with spaCy's NER system:

Real NER systems output entity spans that we can convert to BIO format for analysis or evaluation:

The BIO representation supports token-level evaluation metrics, comparison across different taggers, and training data preparation for sequence models.

Converting spaCy output to BIO format serves a broader purpose: interoperability. Different NLP tools use different internal representations for entity annotations. spaCy uses character-offset spans with Python objects. Stanford NER uses whitespace-delimited CoNLL-style BIO text files. Hugging Face models produce token-level logits that require a decoding step. A shared BIO format provides a common currency for comparing outputs, combining predictions from multiple systems, and routing data through evaluation pipelines regardless of which library produced it.

Modern NER systems built on BERT and similar transformers require a careful treatment of BIO tags due to subword tokenization. BERT's WordPiece tokenizer splits words into subword units, so "Washington" might become ["Washington"] or ["Wash", "##ington"] depending on whether it appears in the model's vocabulary. The "##" prefix marks continuation pieces of the same word.

This creates an alignment problem: your BIO tags are defined over words, but the model processes subword tokens. The standard approach is to assign the word's BIO tag to the first subword piece and a special label, typically "-100" in Hugging Face terminology, to all subsequent pieces. The loss function ignores positions labeled "-100," so the model only predicts tags for the first piece of each word.

This alignment procedure ensures that the model makes one prediction per word (from the first subword) and that subword boundaries do not corrupt the BIO sequence. Without proper alignment, a model might produce a BIO sequence over subword units that is longer than the word sequence, making it impossible to recover coherent entity spans.

The alignment also affects how you interpret predictions at inference time. After the model predicts labels for all subword positions, you collect only the labels at first-subword positions (where word_ids changes to a new value) and discard continuation labels. These first-subword labels form the word-level BIO sequence, which you can then decode using the standard bio_to_spans function we implemented earlier. The subword processing is a preprocessing and postprocessing layer wrapped around the same core BIO machinery.

BIO tags are only as consistent as the annotation guidelines that define what counts as an entity and where its boundaries lie. Ambiguous guidelines produce inconsistent tags, which in turn produce models that learn unreliable boundary conventions.

Several boundary ambiguities recur across NER projects. One common case involves possessive suffixes: should "IBM's" be tagged as B-ORG and the apostrophe-s excluded, or should the full string "IBM's" be the entity? Different datasets make different choices, and a model trained on one convention will have lower precision or recall when evaluated on data annotated with a different one. Tokenization choices interact here: if the tokenizer splits "IBM's" into ["IBM", "'s"], the question is whether "'s" gets I-ORG or O.

Another frequent ambiguity involves titles and honorifics. In "President Biden visited Paris," should "President Biden" be the person entity (including the title) or just "Biden"? The CoNLL-2003 English dataset typically excludes titles, but other datasets include them. Models trained on CoNLL-2003 data will therefore systematically miss the title token when applied to data where titles are included in the span.

Prepositional attachment creates similar challenges. "Bank of America" is clearly a single ORG entity. But "University of California at Berkeley" might be annotated as one span or as "University of California" with "Berkeley" as a separate LOC, depending on how the annotation guidelines treat prepositional phrases. Fine-grained guidelines that address these cases explicitly produce more consistent training data and better-calibrated evaluation results.

These ambiguity examples are not edge cases to be dismissed. They represent systematic differences that produce measurably different model behavior. A model trained on narrow-span annotations for person names (excluding titles) will have systematically lower recall when applied to text where broad-span conventions are expected. Annotation consistency within a dataset is at least as important as annotation quality at any single example.

BIO tagging is the dominant approach for sequence labeling, but it has limitations worth understanding before choosing it for a new application.

The fundamental constraint is that standard BIO assumes non-overlapping entities. Each token receives exactly one tag, so nested or overlapping annotations cannot be represented directly. The workarounds we discussed, including multiple layers, combined tags, and separate passes, add complexity and may not suit all applications. For domains with extensive nesting, such as biomedical text where gene mentions overlap with protein mentions, span-based or graph-based approaches may be more appropriate.

Boundary precision is another challenge. Models often predict the correct entity type but miss exact boundaries. The sentence "the New York Stock Exchange" might yield predictions starting at "New" when guidelines say the entity begins at "the." BIO's token-level representation means every boundary error affects multiple labels simultaneously. BIOES mitigates this slightly by making endpoints explicit, but the underlying challenge remains. Training data annotation guidelines must be precise about boundary rules, otherwise the model learns inconsistent patterns.

Long entities pose particular difficulties for sequence models. An entity spanning ten tokens requires the model to maintain consistent predictions across all ten positions. In BIO, a single mistake, predicting O instead of I in the middle, breaks the entity into two fragments. CRF layers and constrained decoding help by enforcing valid transitions, but very long entities remain error-prone. The Conditional Random Fields chapter will show how to incorporate transition constraints directly into training.

The class imbalance problem is structural and unavoidable with BIO tagging. Because O tokens dominate in every NLP text, models face constant pressure to default to O predictions. Addressing this requires deliberate choices about loss weighting, data sampling, or label-smoothing strategies. There is no free lunch: every technique that helps the model pay more attention to entity tokens risks overfitting to rare types or producing false positives on common non-entity words that superficially resemble entity tokens.

The interaction between tokenization and BIO tagging deserves special mention as a source of subtle bugs. Different tokenizers produce different splits of the same surface text, and the BIO sequence for a sentence differs depending on which tokenizer was used. A model trained with one tokenizer applied to data tokenized with a different tokenizer will have alignment problems that can silently degrade accuracy without producing obvious errors. When combining data from multiple sources or switching tokenizers between training and inference, always verify the alignment between BIO tags and token sequences.

Despite these limitations, BIO tagging works remarkably well in practice. Its simple format, broad tooling support, and compatibility with sequence models like BiLSTM-CRF and BERT-based taggers make it the right choice for most NER applications. The format has been validated on hundreds of benchmarks across dozens of languages and dozens of entity type systems. Its decades of use mean that nearly every NLP library includes built-in support for reading, writing, validating, and evaluating BIO-tagged data. Understanding when and why it fails helps you design better systems and interpret evaluation results more accurately.

BIO tagging provides a standardized format for representing entity boundaries in sequence labeling tasks. The key concepts from this chapter:

The BIO scheme uses three prefixes: B (beginning) marks the first token of an entity, I (inside) marks continuation tokens, and O (outside) marks non-entity tokens. This encoding unambiguously represents entity boundaries, handling adjacent same-type entities correctly. The scheme's success stems from its completeness: three bits of information are both necessary and sufficient to reconstruct any flat entity annotation.

Extended schemes like BIOES add explicit end markers (E) and single-token markers (S) for stronger supervision and easier validation. BIOES trades a slightly larger label vocabulary for clearer boundary signals, which research has shown produces modest but consistent accuracy improvements in boundary-sensitive evaluations. The choice between BIO and BIOES involves a tradeoff between simplicity and boundary precision.

Conversion utilities transform between span annotations and per-token BIO tags. Reliable converters handle edge cases like single-token entities, adjacent entities, and sequence boundaries. Decoders must gracefully handle malformed sequences from model predictions, applying recovery strategies that maximize recall for downstream error analysis.

Validation and repair catch common errors like orphan I tags and type mismatches. Repair strategies can automatically fix many issues, improving downstream usability. Logging every repair is as important as applying it, because the repair log reveals systemic problems in annotation pipelines or model behavior.

Multi-label scenarios require extensions like multiple tag columns or layered encoding for nested entities. Standard BIO assumes non-overlapping annotations, and domains with extensive nesting, such as biomedical NER, often benefit from span-based approaches that handle overlaps natively.

Evaluation at the span level requires exact matching of both boundaries and type. A partial boundary error scores zero, which motivates careful attention to boundary conventions in annotation guidelines and BIOES encoding for models where boundary precision is critical.

Transformer integration requires aligning word-level BIO tags with subword tokenization by labeling only the first subword piece of each word and ignoring continuation pieces during loss computation. This alignment step is a required preprocessing component for any BERT-style NER system.

The next chapters apply BIO tagging to chunking and introduce the probabilistic models, Hidden Markov Models and Conditional Random Fields, that power production sequence labeling systems.

When working with BIO tagging utilities, these parameters control conversion and validation behavior:

• tokens: List of string tokens representing the input sequence. Must align with span indices for correct conversion.
• spans: List of tuples in (start_idx, end_idx, entity_type) format. Uses Python's exclusive end convention where end_idx points to the position after the last entity token.
• tags: List of BIO tag strings, one per token. Valid formats are B-TYPE, I-TYPE, and O.
• entity_type: String identifier for the entity category such as PER, LOC, or ORG. Appears as the suffix after the hyphen in BIO tags.

For BIOES conversion, two additional prefixes apply:

• S-TYPE: Marks single-token entities that do not need a B/I/E structure.
• E-TYPE: Marks the final token of multi-token entities.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about BIO tagging for sequence labeling.