Part-of-Speech Tagging › Rule-Based Taggers
Language AI HandbookAll chapters Part-of-Speech Tagging › Rule-Based TaggersOne section · Sections in this chapter6 / 18 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The earliest POS taggers were purely rule-based systems. Linguists hand-crafted disambiguation rules based on their knowledge of English grammar.
The ENGTWOL System
The most influential rule-based tagger was ENGTWOL (later called the Brill tagger's predecessor), developed at Helsinki in the 1980s. The approach was:
- Look up each word in a lexicon to get its set of possible tags
- Apply constraint rules that eliminate impossible tags based on context
For example, a rule might say: "If a word can be DT (determiner) or JJ (adjective) and the next word is a noun, then it is DT." These rules were written by hand by linguists with deep grammar knowledge.
Rule-based systems have a significant advantage: their decisions are interpretable. You can always trace why the tagger made a particular decision to a specific rule. They also perform well on well-formed text where grammatical rules hold cleanly.
The disadvantage is labor intensity: writing complete rule sets for English takes years of expert effort, and maintaining them as language evolves is costly. Rule-based systems also struggle with informal text where grammatical rules break down.
The ENGTWOL system relied on a two-stage architecture. In the first stage, a morphological analyzer looked up each word form in a large lexicon and returned the set of all possible morphosyntactic readings. A word like "running" might receive readings as a present participle (VERB PRES-PART), a gerund (NOUN), and an adjective. In the second stage, a Constraint Grammar formalism applied ordered rules to eliminate incompatible readings until (ideally) only one reading remained for each word. The rules expressed constraints like "a word with a determiner reading should not retain that reading if the preceding word already has a determiner reading," which captured the fact that two consecutive determiners are very rare in English.
The strength of this approach was its modularity. Rules could be added, removed, or reordered to fix specific error types without disrupting the rest of the system. Linguists maintaining the system could reason about it at a level of abstraction that matched their grammatical intuitions. When the system made an error, debugging was straightforward: find the rule that should have applied but did not, or find the rule that applied when it should not have.
The Brill Tagger
The Brill tagger (1992) made a breakthrough by combining rule-based ideas with corpus learning. Instead of hand-writing rules, it learned transformation rules automatically from annotated data.
The algorithm works in two phases:
Phase 1 (Initial tagging): Assign each word its most frequent tag based on a lexicon learned from the training corpus. Words not seen in training get a default tag based on capitalization and suffix.
Phase 2 (Rule learning): Iteratively learn error-correcting transformation rules. Each rule has the form: "Change tag to tag when condition holds." The algorithm finds the rule that corrects the most errors on the training data, applies it, and repeats.
from collections import Counter, defaultdict
class SimpleBrillTagger:
"""Simplified Brill-style tagger demonstrating the key ideas."""
def __init__(self):
self.word_tag_counts = defaultdict(Counter)
self.most_common_tag = {}
self.default_tag = "NN" # Nouns are most common for unknown words
self.rules = []
def train_baseline(self, tagged_sentences):
"""Phase 1: Learn most-frequent tag per word."""
for sentence in tagged_sentences:
for word, tag in sentence:
self.word_tag_counts[word.lower()][tag] += 1
for word, counts in self.word_tag_counts.items():
self.most_common_tag[word] = counts.most_common(1)[0][0]
def tag_baseline(self, sentence):
"""Apply baseline: most frequent tag per word."""
tagged = []
for word in sentence:
tag = self.most_common_tag.get(word.lower(), self.default_tag)
tagged.append((word, tag))
return tagged
def apply_rule(self, tagged_sentence, condition_tag, new_tag, context_fn):
"""Apply a transformation rule to a tagged sentence."""
result = []
for i, (word, tag) in enumerate(tagged_sentence):
if tag == condition_tag and context_fn(tagged_sentence, i):
result.append((word, new_tag))
else:
result.append((word, tag))
return resultBrill Tagger Concept: Error-Correcting Rules ======================================================= Example sentence: 'I can can the tomatoes' Baseline (most-frequent tag): I PRP can MD can MD the DT tomatoes NNS After rule: 'Change MD to VB when preceded by MD': I PRP can MD can VB the DT tomatoes NNS Error corrected: second 'can' is now tagged VB (base verb), not MD (modal).
The Brill tagger typically achieves accuracy around 95-97% on the Penn Treebank test set, competitive with more complex statistical approaches of its era. Its learned rules are interpretable and can be inspected by linguists.
What made the Brill tagger historically important was the insight that the gap between statistical learning and symbolic reasoning could be bridged. The learned transformation rules looked like linguistically motivated rules: "change VBD to VBN when the previous tag is MD or HVZ" is a rule about perfect aspect that any syntactician would recognize. This interpretability distinguished Brill's approach from the black-box statistical models that were also gaining traction in the early 1990s. It showed that you could have both: rules that are learned automatically from data and that remain interpretable after learning.
The Brill tagger's main limitation is its greedy learning algorithm. At each step, it finds the single best transformation rule and applies it, then finds the next best, and so on. This greedy strategy does not guarantee finding the globally optimal rule set. Two rules might interact synergistically (rule A enables rule B to apply to more cases), but a greedy algorithm might choose a different rule A' that is locally better but prevents B from working. Dynamic programming approaches can find better solutions but at higher computational cost.