Global Tokens › Task-Specific Global Tokens
Language AI HandbookAll chapters Global Tokens › Task-Specific Global TokensOne section · Sections in this chapter4 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
While CLS provides classification-focused global attention, different tasks benefit from different global token configurations. The key insight here is that the optimal placement of global tokens is a function of the task's information requirements, and those requirements are often apparent from the task structure itself.
Question answering, for instance, naturally has two distinct segments: the question and the context passage. The question defines what information is relevant; the context contains candidate answer spans. Every word in the context must, in some sense, evaluate itself against the question: does this token or phrase answer what was asked? That evaluation is precisely what attention enables. Making all question tokens global ensures that every context token can directly attend to the question in a single hop, without having to route through local windows that may not include any question tokens.
This is a fundamentally different design philosophy from the single-CLS approach. Rather than designating one universal aggregator, we designate an entire semantically meaningful segment as globally visible. The architectural choice reflects the task structure: in QA, the question is the global context, and the context passage is the local content being searched. Making question tokens global encodes that semantic relationship directly into the attention mask.
Named entity recognition and coreference resolution benefit similarly. In coreference resolution, a pronoun at position 3000 in a document might refer to a noun phrase at position 100. If the noun phrase tokens are made globally visible, the pronoun token can directly attend to them regardless of the distance. The alternative, waiting for the sliding window to propagate the noun phrase's representation to position 3000, requires many hops and risks losing the specific lexical information needed to resolve the reference.
def create_qa_attention_mask(question_len, context_len, window_size):
"""
Create attention mask for question answering.
Question tokens are global, context tokens use sliding window.
"""
total_len = question_len + context_len
mask = np.zeros((total_len, total_len), dtype=bool)
# Question tokens are global (positions 0 to question_len-1)
for i in range(question_len):
mask[i, :] = True # Question attends to all
mask[:, i] = True # All attend to question
# Context tokens use sliding window among themselves
for i in range(question_len, total_len):
start = max(question_len, i - window_size // 2)
end = min(total_len, i + window_size // 2 + 1)
mask[i, start:end] = True
return mask
# Example: 4 question tokens, 12 context tokens
question_len = 4
context_len = 12
window_size = 4
qa_mask = create_qa_attention_mask(question_len, context_len, window_size)
The mask shows a block pattern. The upper-left quadrant is fully connected (question tokens attending to each other). The left columns and top rows extend this connectivity to context tokens. The lower-right shows the sliding window pattern among context tokens.
This task-specific design makes semantic sense. When answering a question, every context word should be able to consider the question directly. Without global question tokens, a context word might need to hop through multiple windows before accessing question information.
Notice also what happens to the question tokens themselves. They attend to each other fully (the upper-left block), and they attend to all context tokens (the top rows). This means question tokens can aggregate information from the entire context passage, not just from nearby positions. A question token representing "where" can attend directly to the answer span wherever it appears in the context, pulling its representation into the question token's key-value space. The result is that question tokens and context tokens both have mutual global visibility, which is exactly the right structure for extractive QA.