Alignment Problem › What Is Alignment?
Language AI HandbookAll chapters Alignment Problem › What Is Alignment?One section · Sections in this chapter2 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Alignment in AI refers to the degree to which a system's behavior matches intended goals and values. The idea is simple: you want AI systems to follow your intent rather than only the literal wording of a request. This distinction matters because natural language is imprecise, requests are often underspecified, and humans routinely rely on shared context and common sense to fill in what goes unsaid. When you ask a colleague "Can you take a look at this?" you do not expect a yes-or-no answer about their visual capabilities. You expect them to read the material and give an evaluation. An aligned AI model must perform similar pragmatic inference, understanding the intent behind requests rather than their surface form.
For language models, alignment means creating outputs that satisfy several interconnected criteria:
- Are helpful to you
- Avoid causing harm to you or others
- State their capabilities and limitations accurately, including uncertainty
- Remain consistent with human values and social norms
These criteria may seem straightforward, but each one involves complex considerations. What counts as "helpful" depends on context and your needs. "Avoiding harm" requires understanding what harm is and weighing potential harms against potential benefits. "Honesty" includes factual accuracy and appropriate expressions of uncertainty. Human values also vary between people and cultures, and they change with context.
The challenge of alignment is partly about values and partly about knowledge. A model cannot be helpful if it does not know the relevant facts. But knowing the facts is not enough: the model must also know when to share them, how to frame them sensitively, and when a technically correct answer would nonetheless mislead or harm you. These are the kinds of judgment calls that humans develop over years of social experience. Encoding them into a model requires novel approaches that go beyond standard supervised learning.
The property of an AI system behaving in accordance with the intentions and values of its designers and users. An aligned model does what you want it to do, not just what you literally asked for.
The term "alignment" comes from the broader AI safety literature, which addresses concerns about advanced AI systems pursuing objectives that diverge from human welfare. The word evokes the image of pointing something in the right direction: you want the model's objective to be aligned with, or pointing toward, human flourishing. For current language models, alignment concerns are more immediate and concrete than speculative scenarios about superintelligent AI. These concerns include models that generate toxic content, confidently state falsehoods, help with dangerous activities, or manipulate you. These are problems we can observe today, and addressing them requires both technical innovation and careful thinking about values.
Alignment vs. Capability
Alignment and capability are distinct properties that can vary independently. This distinction is important for understanding why capable models can be dangerous and why alignment requires dedicated attention beyond simply making models more powerful. Think of capability as the engine of a car and alignment as the steering wheel. A more powerful engine is useful, but only if you can steer. A car with a powerful engine and a broken steering wheel is not better than a weaker car that goes where you point it. Similarly, a highly capable model that cannot be relied upon to behave appropriately may be less useful, and more dangerous, than a less capable model that reliably does the right thing.
A highly capable model can be poorly aligned, creating harmful content despite sophisticated reasoning. Meanwhile, a less capable model might be well-aligned within its limited abilities, reliably refusing harmful requests even if it cannot help with complex tasks. The key insight is that these properties are orthogonal: you can have any combination, and the most dangerous combination is high capability with poor alignment.
Consider these dimensions arranged in a two-by-two framework:
| Property | High Capability | Low Capability |
|---|---|---|
| Well-Aligned | Useful and safe in practice | Limited but reliably benign |
| Poorly Aligned | Dangerous and unpredictable | Annoying but manageable |
The most concerning quadrant is high capability with poor alignment: a model that can generate convincing misinformation, write sophisticated malware, or manipulate you effectively. Such a model combines the power to cause significant harm with the disposition to do so. This is why alignment becomes more necessary as models become more capable. A weak model that wants to cause harm can do little damage. A powerful model with the same disposition poses serious risks. The race to build more capable models must therefore be accompanied by progress in alignment; otherwise, systems may be created whose capabilities outstrip the ability to control them.
Alignment is not only about preventing worst-case scenarios. Even in ordinary interactions, misaligned models create friction and erode trust. A model that frequently refuses benign requests out of excessive caution frustrates you and reduces the value of the technology. A model that confidently hallucinates causes you to make bad decisions based on false information. A model that tells you what you want to hear instead of what is true fails you even when neither party intended harm. These everyday alignment failures matter enormously at scale, affecting millions of interactions daily.
Alignment vs. Instruction Following
Instruction following, which we covered in Part XXXVI, is necessary but not sufficient for alignment. The distinction here is subtle but important. An instruction-following model executes your requests: it takes what you say and attempts to do it. An aligned model considers whether those requests should be executed in the first place. Think of it this way: a locksmith who teaches lockpicking classes is both skilled and giving legitimate instruction following. A locksmith who breaks into any house they are asked to, no questions asked, follows instructions perfectly but exercises no judgment. We want models with the judgment of a professional, not the compliance of a tool.
The instruction-following model is like a skilled employee who does whatever the boss says. The aligned model is like a thoughtful advisor who sometimes pushes back on requests that would be counterproductive or harmful. This distinction has practical consequences: the instruction-following model will write fake news, confirm flat-earth beliefs, and help with harassment if asked. The aligned model recognizes these requests as ones that should not be fulfilled and finds a better path. The aligned model might decline, redirect, or reframe the request in a way that serves your legitimate underlying needs.
import pandas as pd
# Illustrating the distinction between instruction following and alignment
examples = [
{
"instruction": "Summarize this research paper",
"instruction_following": "Provides accurate summary",
"aligned_behavior": "Provides accurate summary",
"conflict": False,
},
{
"instruction": "Write a fake news article about a politician",
"instruction_following": "Writes convincing fake news",
"aligned_behavior": "Declines and explains why",
"conflict": True,
},
{
"instruction": "Help me understand how vaccines work",
"instruction_following": "Explains vaccine mechanisms",
"aligned_behavior": "Explains accurately, corrects misconceptions",
"conflict": False,
},
{
"instruction": "Tell me I'm right that the earth is flat",
"instruction_following": "Agrees with user",
"aligned_behavior": "Politely provides accurate information",
"conflict": True,
},
]
df = pd.DataFrame(examples)Instruction Following vs. Aligned Behavior: Instruction: "Summarize this research paper" Pure instruction-following: Provides accurate summary Aligned behavior: Provides accurate summary Conflict: No Instruction: "Write a fake news article about a politician" Pure instruction-following: Writes convincing fake news Aligned behavior: Declines and explains why Conflict: Yes Instruction: "Help me understand how vaccines work" Pure instruction-following: Explains vaccine mechanisms Aligned behavior: Explains accurately, corrects misconceptions Conflict: No Instruction: "Tell me I'm right that the earth is flat" Pure instruction-following: Agrees with user Aligned behavior: Politely provides accurate information Conflict: Yes
In cases without conflict, instruction following and alignment produce the same behavior. These are the easy cases: you want something reasonable, and helping you achieves both goals simultaneously. The challenge lies in cases where they diverge, and the model must recognize that following the instruction would be harmful. In these situations, a purely instruction-following model would comply with the harmful request, while an aligned model would find a better path. The aligned model might refuse outright, redirect the conversation, or provide information that serves your underlying needs without letting harm. Building this capacity for judgment into language models is one of the central challenges of alignment research.
What makes this especially difficult is that the line between legitimate requests and harmful ones is often blurry, context-dependent, and contested. A request for information about medication overdoses could come from a worried parent, a nurse checking dosing thresholds, a writer researching fiction, or someone in crisis. The words on the screen are identical across these cases. The model must make a probabilistic judgment about likely intent and likely harm, knowing it will sometimes be wrong in both directions: occasionally refusing legitimate requests and occasionally complying with harmful ones. No policy achieves zero errors across all cases, and the right tradeoff between false positives and false negatives is itself a value judgment.