Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Sentence Meaning and Natural Language Inference › Entailment, Contradiction, and Neutrality

Michael BrenndoerferMay 9, 20259 min read

Language AI HandbookAll chapters

Part 8 of 67Semantics and Information Extraction

Chapter 51Sentence Meaning and Natural Language Inference › Entailment, Contradiction, and Neutrality

Reading section 3 of 12 · Entailment, Contradiction, and Neutrality · 9 minSection 3 of 12 · 9 min

Section 3 of 12 · 9 min read

Sentence Meaning and Natural Language Inference › Entailment, Contradiction, and NeutralitySection view · Sections in this chapter3 / 12 ↓

Sections in this chapter12

Reading tools Definitions, links & notes

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Notes & comments

Notes & commentsLeave notes and join the discussionSelection toolbarShow comment actions when selecting text. Turn off for the browser menu only.Comment indicatorsShow small counts beside annotated passagesPassage highlightsGently shade annotated text selections

NLI is usually framed as a three-way classification problem. Given a premise $p$ and a hypothesis $h$ , predict one of three labels:

Entailment: if $p$ is true, then $h$ must be true.
Contradiction: if $p$ is true, then $h$ must be false.
Neutral: $p$ neither guarantees $h$ nor rules it out.

The word "must" is doing important work in the first two definitions. It signals that the relation holds in virtue of meaning, not in virtue of a lucky coincidence in the actual world. The premise is treated as given, and the question is what that premise forces.

Using the truth-conditional view, we can write these relations in terms of sets of worlds.

Entailment

A premise $p$ entails a hypothesis $h$ , written $p \models h$ , if every world in which $p$ is true is also a world in which $h$ is true:

p \models h \quad \text{iff} \quad \mathcal{M}(p) \subseteq \mathcal{M}(h)

where:

$p$ : the premise sentence
$h$ : the hypothesis sentence
$\models$ : the entailment relation between sentences
$\mathcal{M}(p)$ : the set of worlds in which $p$ is true
$\mathcal{M}(h)$ : the set of worlds in which $h$ is true

Entailment is directional. If $p$ entails $h$ , it does not follow that $h$ entails $p$ .

The subset relation formalizes the intuition that entailment makes the hypothesis's truth a necessary consequence of the premise's truth. The set of worlds satisfying the premise is "smaller" in the sense that all of its members also satisfy the hypothesis. Nothing about entailment requires the two sentences to share words, which is why entailment can be carried by paraphrase, by lexical relations, or by world knowledge.

Contradiction

A premise $p$ contradicts a hypothesis $h$ if there is no possible world in which both are true:

\mathcal{M}(p) \cap \mathcal{M}(h) = \emptyset

where:

$p$ : the premise sentence
$h$ : the hypothesis sentence
$\mathcal{M}(p)$ : the set of worlds in which $p$ is true
$\mathcal{M}(h)$ : the set of worlds in which $h$ is true
$\cap$ : set intersection
$\emptyset$ : the empty set, meaning no world makes both sentences true

Contradiction is symmetric in the sense that if $p$ contradicts $h$ , then $h$ contradicts $p$ .

The symmetry of contradiction follows directly from the symmetry of set intersection: the intersection of two sets does not depend on the order in which you name them, so the empty set result is the same either way. This is a clean structural fact that will be useful when we test models for directional sensitivity.

Neutrality is the residual case: the premise does not entail the hypothesis, and the two are not contradictory. In set terms, there is at least one world where both are true, but the premise's set of worlds is not contained in the hypothesis's set. Informally, the premise leaves the hypothesis undecided. It may be true, it may be false, but the premise alone does not settle it. Neutrality is a residual category after entailment and contradiction are ruled out. Its mixture of cases complicates interpretation; which label is hardest for a model is an empirical question.

A subtlety: classical logic makes entailment vacuous when the premise is impossible. If $\mathcal{M}(p) = \emptyset$ , then $\mathcal{M}(p) \subseteq \mathcal{M}(h)$ for every $h$ , because the empty set is a subset of every set. Under that definition, "The moon is made of green cheese and not made of green cheese" would entail every sentence whatsoever. For the three labels to be mutually exclusive under these classical definitions, we assume $\mathcal{M}(p) \neq \emptyset$ . Treating a dataset premise as given does not establish that it is logically consistent. The three-way account used here therefore excludes inconsistent premises. A dataset containing them needs an explicit policy beyond these mutually exclusive labels.

Before moving to directional examples, it helps to see the three relations as signatures over possible worlds. The heatmap below encodes whether the premise and hypothesis are true or false in each toy world; entailment forbids premise-only worlds, contradiction forbids shared worlds, and neutrality requires a shared world and a premise-only world.

VisualizationOut [4]
CollapseExpand

Heatmap of premise and hypothesis truth across toy worlds for three NLI relations. — Possible-world signatures for entailment, contradiction, and neutrality across ten toy worlds. Each column is a toy world, and colors separate worlds where only the premise is true, only the hypothesis is true, both are true, or neither is true; entailment forbids premise-only worlds, contradiction forbids shared worlds, and neutrality requires both a shared world and a premise-only world.

Entailment Is Not SymmetricLink Copied

The direction of inference matters. Consider:

Premise: "A poodle barked all night."
Hypothesis: "A dog barked all night."

This is entailment. A poodle is a dog, so any world where a poodle barked is a world where a dog barked. The premise is logically stronger: it commits to a more specific kind of dog. But reverse the pair:

Premise: "A dog barked all night."
Hypothesis: "A poodle barked all night."

This is neutral. The dog could be a Labrador, a beagle, or a mixed breed. The premise does not guarantee that the dog was a poodle. It is compatible with the hypothesis, and it is also compatible with the hypothesis being false.

This asymmetry is built into the task definition. The premise is evidence. The hypothesis is a claim to be checked against that evidence. Swapping the two changes the question, sometimes dramatically. A model that ignores direction, for instance by comparing the two sentences symmetrically, will systematically blur the distinction between "the premise is more specific" and "the premise is more general."

Contradiction Is Not the Same as FalsehoodLink Copied

A hypothesis can be false in the actual world without contradicting the premise. Suppose an unspecified integer $n$ is even. Keep its value fixed within each premise-hypothesis pair, while considering the integer values compatible with the premise:

The number is even.

The hypothesis:

The number is prime.

In the actual world, some even numbers are prime (2 is both), and some are not (4 is even but not prime). The premise does not entail the hypothesis, and the two are not contradictory, because there is at least one possible world, one where the number is 2, in which both are true. The label is neutral. Now change the hypothesis:

The number is odd.

This contradicts the premise. No integer is both even and odd under the standard definitions. The exclusion follows from those truth conditions and holds for every integer; a statistical association between the words does not establish it.

A model might place "even" and "odd" near each other because they occur in similar contexts. Distributional similarity alone does not establish their logical relation. The representations from Part III and Part IV remain useful, but evaluating inference requires checking the relation expressed by the whole pair.

Neutral Is Not "Unknown"Link Copied

Neutral does not mean the model is ignorant or the annotator was unsure. It means the premise does not determine the hypothesis. For example:

Premise: "A woman is playing the violin."
Hypothesis: "A woman is performing in an orchestra."

The premise is compatible with the hypothesis. The woman could be in an orchestra. But she could also be practicing alone, playing in a jazz trio, or recording in a studio. The premise does not entail the hypothesis, and it does not contradict it. Neutral is the correct label even if, in the actual world, the woman happens to be in an orchestra. The label is about the inference from premise to hypothesis, not about the truth of either sentence on its own.

This is a common source of confusion. NLI is not fact-checking. It asks what follows from the premise, not what is true in the real world. A premise can be false and still entail something; a hypothesis can be true and still be merely neutral with respect to the premise. Keeping these apart is what makes NLI a semantic task rather than a database lookup.

Natural Language Inference versus Logical EntailmentLink Copied

The applied NLI task can use a looser criterion than formal logical entailment. In the RTE-2 task definition, a human reading the premise would typically infer that the hypothesis is most likely true. That operational wording assumes ordinary language understanding and background knowledge; it is not a calibrated conditional probability or a restriction to formal logical consequence.

That looseness is practical, but it creates ambiguity. Consider:

Premise: "The Eiffel Tower is in Paris."
Hypothesis: "The Eiffel Tower is in France."

Strictly speaking, this requires geographical knowledge: Paris is in France. Under a labeling policy that permits this background fact, the relation is entailment. A purely textual system that only knows co-occurrence may also call it entailment, but for the wrong reason. The label is the same; the inference process is different. One system reasons from geography, the other from a statistical habit picked up from the corpus.

The lesson is that NLI labels are annotation targets, not Platonic logical relations. They reflect a mixture of semantics, world knowledge, and annotator judgment. Different annotation guidelines can push the same example into different label buckets, and models trained on such data inherit those choices. Robust evaluation has to keep that in mind, and it has to report the label semantics alongside the accuracy number.

Comments

Add to the conversation

A question, a correction, or an example from your own work.

No comments yet. Be the first to share your thoughts!