Sentence Meaning and Natural Language Inference

Consider two sentences:

The cat chased the dog.

The dog chased the cat.

A bag-of-words representation from Part II gives these sentences the same vector when punctuation is separated from words and both use the same word-count vocabulary. Yet the sentences assign different roles to the two animals. Neither entails the other, and both can be true: the animals might chase each other. Reversing who did what changes the proposition without necessarily creating a contradiction. Sentence meaning requires structure as well as word meanings.

The main lesson of this chapter is that the gap between "same words" and "same meaning" is exactly where sentence semantics lives. Every subsequent topic, from entailment to quantifier scope to model evaluation, is a way of pinning down that gap precisely enough to reason about it, and precisely enough to test whether a machine has learned it.

Natural language inference, or NLI, is the task of making that relational structure explicit. Given a premise and a hypothesis, an NLI system decides whether the premise entails the hypothesis, contradicts it, or neither entails nor contradicts it. The premise functions as evidence, and the hypothesis functions as a claim to be checked against that evidence. For example:

• Premise: "A poodle barked all night." Hypothesis: "A dog barked all night." This is entailment, because a poodle is a kind of dog, so any situation that makes the premise true also makes the hypothesis true.
• Premise: "The door is open." Hypothesis: "The door is closed." This is contradiction, because a single door cannot be fully open and closed at the same time in the ordinary sense of those words.
• Premise: "A woman is playing the violin." Hypothesis: "A woman is performing in an orchestra." This is neutral, because the premise leaves the setting undetermined: the woman might be in an orchestra, or she might be practicing alone.

NLI provides one way to test sensitivity to negation, quantification, lexical relations, event structure, and scope. The same premise-hypothesis interface also supports concrete applications. SummaC segments source documents and summaries into sentences, applies NLI to sentence pairs, and aggregates their scores to detect source inconsistency. Other work uses NLI for evidence retrieval and reranking, factual-consistency evaluation, and filtering synthetic training examples. Entailment is a different target from relevance: a passage can be relevant precisely because it contradicts a claim. In a retrieval-augmented answer system, checking a claim against retrieved evidence would still leave evidence quality and real-world truth to be established separately. Within Part VIII: Semantics and Information Extraction, this chapter connects word-level meaning from lexical semantics to the predicate-argument structure and meaning representations in the chapters that follow.

Throughout the hand-labeled examples, hold referents, relevant times, and senses fixed when descriptions are intended to corefer. Examples of lexical ambiguity explicitly use different senses. Open/closed examples use a binary status unless stated otherwise. These conventions prevent an unstated change of referent, event, or time from changing a label.

The chapter has five goals:

1. Explain what sentence meaning is, beyond word meaning.
2. Define entailment, contradiction, and neutrality precisely enough to evaluate models.
3. Distinguish entailment from presupposition, implicature, and other meaning layers.
4. Show how NLI datasets and models work, and why robust evaluation is difficult.
5. Build a small NLI battery in code and use it to expose strengths and failure modes.

Sentence meaning combines word meanings according to their structure. The principle of compositionality makes that relationship explicit.

The key phrase in that principle is "the rules used to combine them." Word meanings alone are not enough, because the same words can be assembled in more than one way. Systematic combination rules help explain how readers interpret sentences they have never encountered as complete expressions. This is one motivation for compositionality, rather than a claim that syntax and a list of literal word meanings account for every contextual interpretation.

Compositionality explains why "The cat chased the dog" and "The dog chased the cat" differ despite sharing the same words. The words contribute entities and relations, while their combination assigns the roles. In "The cat did not chase the dog," negation denies the proposition that the cat chased the dog; it does not replace the dog with a non-dog. The verb phrase describes a property before its subject is supplied. In "The cat chased the dog and the dog chased the cat," the coordinating "and" joins two propositions about the same two animals in reversed roles. Conjunction alone does not determine whether we describe the chasing as separate episodes or parts of one larger event.

A useful way to formalize sentence meaning is through truth conditions. The truth-conditional approach does not ask what a sentence "makes you picture." It asks a sharper question: in which situations would this sentence be true? Let $\mathcal{W}$ be a set of possible situations or worlds. A sentence $s$ denotes a set of worlds:

where:

• $\mathcal{M}(s)$: the set of worlds in which sentence $s$ is true
• $\mathcal{W}$: the set of all possible worlds under consideration
• $\subseteq$: set inclusion, so $\mathcal{M}(s)$ is a subset of $\mathcal{W}$

The set $\mathcal{M}(s)$ contains exactly those worlds in which $s$ is true. This is a simple but useful idea, because it turns meaning into an object we can manipulate with ordinary set operations:

• "The cat is on the mat" denotes the set of worlds where the cat is on the mat.
• "The cat is not on the mat" denotes the complement of that set.
• "The cat is on the mat and the dog is barking" denotes the intersection of two sets.
• "The cat is on the mat or the dog is barking" denotes the union of two sets.
• "Every cat is on a mat" denotes worlds where the set of cats is a subset of the set of things on mats.

In this simplified classical model, each sentence is true or false at every considered world, and "or" is inclusive. Negation is then complement within $\mathcal{W}$, conjunction is intersection, and disjunction is union. Universal quantification can be expressed as inclusion between predicate extensions within a world, as we will see below. Presupposition failures that lack a truth value require a richer semantics; the complement rule does not silently cover those cases.

This truth-conditional view does not capture everything about meaning. It does not directly capture emphasis, sarcasm, social meaning, or the conversational suggestion that "some" can convey "not all." Two sentences can have the same truth conditions yet feel different, as when one is more polite, more emphatic, or more informative. But it gives us a precise core for entailment and contradiction, and that core is enough to make NLI a well-defined task.

Structure is not optional. The sentence "I saw the man with the telescope" has at least two readings:

• I used a telescope to see the man.
• I saw the man associated with or in possession of the telescope.

The words are the same, but the syntactic attachment differs. In the first reading, "with the telescope" describes the instrument of seeing. In the second, it modifies "man," associating the telescope with him. Physical holding is one possible interpretation, not an entailment of noun attachment: his telescope could be mounted on a tripod beside him. The two attachments therefore support different inferences. This is one reason the syntax chapters in Part VII matter for semantics.

Context matters too. "She is here" depends on who "she" refers to and where "here" is. "The meeting is tomorrow" depends on the utterance time. Lexical ambiguity is related but distinct: "bank" has financial and river-edge senses, while the local phrase "raised rates" already selects the financial sense in ordinary literal use. A shorter phrase such as "near the bank" may need more context. Sentence-pair datasets such as SNLI provide much less discourse than a complete conversation or document. Success on isolated pairs need not transfer to passages where pronouns and definite descriptions depend on earlier text. Coreference and discourse relations return in Part IX.

For now, keep three levels separate:

• Sentence meaning: truth conditions and entailment relations. This is what the sentence says once lexical senses, contextual referents, and relevant times have been fixed.
• Speaker meaning: what a speaker intends to convey in context. This includes implicature, irony, emphasis, and all the ways an utterance can mean more than its literal content.
• World knowledge: facts about the actual world that help decide whether a relation holds. Knowing that Paris is in France lets you judge that "The Eiffel Tower is in Paris" entails "The Eiffel Tower is in France," even though nothing in the words themselves encodes that geography.

An NLI judgment can depend on more than one of these levels. Recognizing which level a given example depends on is the first step toward interpreting a model's errors correctly.

NLI is usually framed as a three-way classification problem. Given a premise $p$ and a hypothesis $h$, predict one of three labels:

• Entailment: if $p$ is true, then $h$ must be true.
• Contradiction: if $p$ is true, then $h$ must be false.
• Neutral: $p$ neither guarantees $h$ nor rules it out.

The word "must" is doing important work in the first two definitions. It signals that the relation holds in virtue of meaning, not in virtue of a lucky coincidence in the actual world. The premise is treated as given, and the question is what that premise forces.

Using the truth-conditional view, we can write these relations in terms of sets of worlds.

The subset relation formalizes the intuition that entailment makes the hypothesis's truth a necessary consequence of the premise's truth. The set of worlds satisfying the premise is "smaller" in the sense that all of its members also satisfy the hypothesis. Nothing about entailment requires the two sentences to share words, which is why entailment can be carried by paraphrase, by lexical relations, or by world knowledge.

The symmetry of contradiction follows directly from the symmetry of set intersection: the intersection of two sets does not depend on the order in which you name them, so the empty set result is the same either way. This is a clean structural fact that will be useful when we test models for directional sensitivity.

Neutrality is the residual case: the premise does not entail the hypothesis, and the two are not contradictory. In set terms, there is at least one world where both are true, but the premise's set of worlds is not contained in the hypothesis's set. Informally, the premise leaves the hypothesis undecided. It may be true, it may be false, but the premise alone does not settle it. Neutrality is a residual category after entailment and contradiction are ruled out. Its mixture of cases complicates interpretation; which label is hardest for a model is an empirical question.

A subtlety: classical logic makes entailment vacuous when the premise is impossible. If $\mathcal{M}(p) = \emptyset$, then $\mathcal{M}(p) \subseteq \mathcal{M}(h)$ for every $h$, because the empty set is a subset of every set. Under that definition, "The moon is made of green cheese and not made of green cheese" would entail every sentence whatsoever. For the three labels to be mutually exclusive under these classical definitions, we assume $\mathcal{M}(p) \neq \emptyset$. Treating a dataset premise as given does not establish that it is logically consistent. The three-way account used here therefore excludes inconsistent premises. A dataset containing them needs an explicit policy beyond these mutually exclusive labels.

Before moving to directional examples, it helps to see the three relations as signatures over possible worlds. The heatmap below encodes whether the premise and hypothesis are true or false in each toy world; entailment forbids premise-only worlds, contradiction forbids shared worlds, and neutrality requires a shared world and a premise-only world.

The direction of inference matters. Consider:

• Premise: "A poodle barked all night."
• Hypothesis: "A dog barked all night."

This is entailment. A poodle is a dog, so any world where a poodle barked is a world where a dog barked. The premise is logically stronger: it commits to a more specific kind of dog. But reverse the pair:

• Premise: "A dog barked all night."
• Hypothesis: "A poodle barked all night."

This is neutral. The dog could be a Labrador, a beagle, or a mixed breed. The premise does not guarantee that the dog was a poodle. It is compatible with the hypothesis, and it is also compatible with the hypothesis being false.

This asymmetry is built into the task definition. The premise is evidence. The hypothesis is a claim to be checked against that evidence. Swapping the two changes the question, sometimes dramatically. A model that ignores direction, for instance by comparing the two sentences symmetrically, will systematically blur the distinction between "the premise is more specific" and "the premise is more general."

A hypothesis can be false in the actual world without contradicting the premise. Suppose an unspecified integer $n$ is even. Keep its value fixed within each premise-hypothesis pair, while considering the integer values compatible with the premise:

The number is even.

The hypothesis:

The number is prime.

In the actual world, some even numbers are prime (2 is both), and some are not (4 is even but not prime). The premise does not entail the hypothesis, and the two are not contradictory, because there is at least one possible world, one where the number is 2, in which both are true. The label is neutral. Now change the hypothesis:

The number is odd.

This contradicts the premise. No integer is both even and odd under the standard definitions. The exclusion follows from those truth conditions and holds for every integer; a statistical association between the words does not establish it.

A model might place "even" and "odd" near each other because they occur in similar contexts. Distributional similarity alone does not establish their logical relation. The representations from Part III and Part IV remain useful, but evaluating inference requires checking the relation expressed by the whole pair.

Neutral does not mean the model is ignorant or the annotator was unsure. It means the premise does not determine the hypothesis. For example:

• Premise: "A woman is playing the violin."
• Hypothesis: "A woman is performing in an orchestra."

The premise is compatible with the hypothesis. The woman could be in an orchestra. But she could also be practicing alone, playing in a jazz trio, or recording in a studio. The premise does not entail the hypothesis, and it does not contradict it. Neutral is the correct label even if, in the actual world, the woman happens to be in an orchestra. The label is about the inference from premise to hypothesis, not about the truth of either sentence on its own.

This is a common source of confusion. NLI is not fact-checking. It asks what follows from the premise, not what is true in the real world. A premise can be false and still entail something; a hypothesis can be true and still be merely neutral with respect to the premise. Keeping these apart is what makes NLI a semantic task rather than a database lookup.

The applied NLI task can use a looser criterion than formal logical entailment. In the RTE-2 task definition, a human reading the premise would typically infer that the hypothesis is most likely true. That operational wording assumes ordinary language understanding and background knowledge; it is not a calibrated conditional probability or a restriction to formal logical consequence.

That looseness is practical, but it creates ambiguity. Consider:

• Premise: "The Eiffel Tower is in Paris."
• Hypothesis: "The Eiffel Tower is in France."

Strictly speaking, this requires geographical knowledge: Paris is in France. Under a labeling policy that permits this background fact, the relation is entailment. A purely textual system that only knows co-occurrence may also call it entailment, but for the wrong reason. The label is the same; the inference process is different. One system reasons from geography, the other from a statistical habit picked up from the corpus.

The lesson is that NLI labels are annotation targets, not Platonic logical relations. They reflect a mixture of semantics, world knowledge, and annotator judgment. Different annotation guidelines can push the same example into different label buckets, and models trained on such data inherit those choices. Robust evaluation has to keep that in mind, and it has to report the label semantics alongside the accuracy number.

Before judging an NLI prediction, identify the meaning relation in the example. Presupposition, conversational implicature, and asserted entailment behave differently, even when an annotation scheme represents them with the same output label.

A presupposition is a background assumption that a sentence takes for granted. It is not the main assertion. When you say "The king of France is bald," the main claim is about the king's hair, but the sentence also quietly assumes that there is a king of France at all. Classic triggers include:

• Definite descriptions: "The king of France" presupposes that there is a king of France.
• Factive verbs: "John knows that the earth is round" presupposes that the earth is round.
• Aspectual verbs: "John stopped smoking" presupposes that John used to smoke.
• Cleft constructions: "It was Mary who broke the vase" presupposes that someone broke the vase.
• Possessives: "Alice's car" backgrounds a salient car associated with Alice; the association need not be ownership.
• Questions: in an ordinary information-seeking use, "Why did John leave?" normally treats John's leaving as given. Such inferences depend on context, and whether they are semantic presuppositions is debated.

In these ordinary readings, the backgrounded content is presented as already available rather than as the main assertion. A listener can still challenge it directly: "There is no king of France." Backgrounding is not immunity from correction.

A useful diagnostic for presupposition is projection: the background inference often survives negation, questioning, and modal embedding. Projection depends on context and embedding structure; it is not an exceptionless rule. This is what distinguishes it from ordinary assertion content, which flips under negation.

If $p$ entails $h$, then $\neg p$ does not generally entail $h$. For example, denying that a poodle barked does not settle whether a dog barked. By contrast, a prior-smoking presupposition often remains available in an ordinary reading of "John did not stop smoking." This projection diagnostic depends on context and embedding. Neither universal cancellation of entailment nor exceptionless projection follows: a tautology, for instance, is entailed by both $p$ and $\neg p$.

Presupposition failure occurs when a required background assumption is not satisfied. If there is no king of France, "The king of France is bald" creates a reference problem. In a Strawsonian analysis, a use of that sentence can fail to make a true or false assertion; this is a theory-specific analysis, not an uncontested treatment of every definite description. The standard three-way NLI alphabet has no separate presupposition-failure label. Annotation instructions therefore need to state how such cases are handled.

This matters in practice. If a document says "The company's new CEO announced layoffs," a reader infers that the company has a new CEO. A question-answering system that ignores this presupposition may miss the existence claim. A factuality evaluation must state whether its support standard covers asserted and presupposed content. Presupposing a proposition does not make it exempt from evidential checking. We will return to presupposition and pragmatics in Part IX.

A conversational implicature is content a speaker conveys beyond what is literally asserted. In Grice's account, listeners use a presumption of cooperation, contextual knowledge, and expectations about informativeness, truthfulness, relevance, and clarity. The scalar examples below concern conversational implicature; conventional implicatures have different properties.

• "Some students passed the exam" often implicates "Not all students passed." The speaker could have said "all," and the choice of the weaker term suggests the stronger one is not warranted.
• "John has at least three children" is compatible with four. Bare numerals such as "John has three children" have competing semantic analyses, so an exactly-three judgment needs an explicit interpretation.

Conversational implicatures are typically cancellable. You can cancel an implicature without contradiction:

Some students passed the exam. In fact, all of them did.

That sounds fine. The implicature is defeated, but the sentence remains coherent. By contrast, you cannot cancel an entailment in the same way. If "A poodle barked" entails "A dog barked," then saying "A poodle barked, but no dog barked" is contradictory. The contrast between cancellable implicature and non-cancellable entailment is one of the sharpest tests we have for distinguishing the two.

Under a strict truth-conditional annotation policy, conversational implicatures should not be treated as entailments. "Some students passed" does not entail "Not all students passed" in strict semantics. An annotation policy must say whether to include that pragmatic enrichment. Otherwise, a disagreement about the policy can look like a model error. Strict logical independence, defeated implicature, and missing context are different reasons a pair might receive a non-entailment label.

Some entailments come from word meaning rather than sentence structure. In the previous chapter on lexical semantics, we saw relations such as hyponymy, hypernymy, synonymy, and antonymy. These relations drive many NLI judgments.

• Hyponymy: "poodle" is a hyponym of "dog." "A poodle barked" entails "A dog barked."
• Hypernymy: "vehicle" is a hypernym of "car." "Alice bought a car" entails "Alice bought a vehicle."
• Antonymy: "open" and "closed" are antonyms in the relevant sense. "The door is open" contradicts "The door is closed."
• Meronymy: "finger" is a part of "hand." "The finger was cut" does not straightforwardly entail "The hand was cut," but it often supports a plausible inference. This is a case where strict entailment and defeasible inference diverge.

Event structure adds another layer. Verbs describe events with participants and roles. "Alice sold Bob a car" entails "Bob bought a car from Alice" because selling and buying are two perspectives on the same transfer event. "Alice gave Bob a book" entails "Bob received a book." "The ice melted" entails "The ice became liquid." These are lexical entailments that depend on the internal structure of the event, not just on set inclusion between nouns. The relation between "sell" and "buy" is not a hypernym relation; it is a relation between two ways of describing a single transaction from opposite viewpoints.

This is a bridge to the next chapter on semantic roles and argument structure. For now, notice that NLI is not only about logic. It is also about lexical semantics, event structure, and world knowledge. A model that masters the logic of "and" and "not" but never learns that selling entails buying will still miss a large and important class of inferences.

Coordination creates entailments that are easy to get wrong. Compare:

• "Alice and Bob left." This entails "Alice left" and "Bob left." The predicate distributes over the conjunction, because leaving is something each person does individually.
• "Alice and Bob lifted the piano." On its collective reading, this does not entail "Alice lifted the piano alone" or "Bob lifted the piano alone." The piano may require their combined effort. An individual can still participate in the joint lift.

The same surface form, "Alice and Bob [verb]ed," supports different inferences depending on the predicate. Some predicates are distributive, some are collective, and some permit either reading in context. Treating every coordinated subject as distributive licenses inferences that the collective reading does not support. Lexical predicate properties and sentence structure both matter.

Compositionality is easiest to see when we treat word meanings as functions. An adjective like "red" can be modeled as a function from entities to truth values, or equivalently as a set of red things. At a fixed world $w$, write $\llbracket e \rrbracket_w$ for the set of entities satisfying a predicate expression $e$. This is an entity set, distinct from the set of worlds $\mathcal{M}(s)$ used for complete sentences. For an intersective use of "red," combining it with "car" intersects the two entity sets:

where:

• $\llbracket \text{red car} \rrbracket_w$: the set of red cars, i.e., the meaning of the phrase "red car"
• $\llbracket \text{red} \rrbracket_w$: the set of red things
• $\llbracket \text{car} \rrbracket_w$: the set of cars
• $\cap$: set intersection, so the phrase denotes things that are both red and cars

This works for intersective adjectives. But not all adjectives are intersective, and the difference has direct consequences for entailment.

• "Former" is non-intersective and introduces a past-time condition. A former president need not hold the relevant presidency now. The phrase is not formed by intersecting presidents with a context-free set of "former things"; offices, times, and tenures matter.
• "Fake" is privative. A fake gun is not a gun in the relevant sense. It is designed to look like one, but the whole point is that it fails the ordinary criteria for being a gun.
• "Big" is subsective and gradable. A big mouse is big for a mouse, not necessarily big in absolute terms. A big mouse is still a mouse, so "big mouse" does entail "mouse," but the size is relative to the noun.

These distinctions matter for NLI. "Alice is a former president" does not entail "Alice is a president." "This is a fake gun" does not entail "This is a gun." A model that treats all adjectives as intersective will make systematic errors, because it will assume that every adjective-noun phrase entails the bare noun. The lexicon has to tell the model which adjectives behave which way.

Quantifiers are another core part of compositional meaning. Generalized quantifier theory treats determiners as relations between sets. This perspective unifies the meanings of "every," "some," "no," and "most" under a single scheme, and it makes their entailment patterns easy to state.

• "Every dog barks": the set of dogs is a subset of the set of barkers.

where:

• $A$: the restrictor set (e.g., the set of dogs)

• $B$: the scope set (e.g., the set of barkers)

• $\mathrm{every}(A)(B)$: true when the determiner "every" relates $A$ and $B$

• $\subseteq$: set inclusion

• "Some dog barks": the set of dogs intersects the set of barkers.

where:

• $A$: the restrictor set (e.g., the set of dogs)

• $B$: the scope set (e.g., the set of barkers)

• $\mathrm{some}(A)(B)$: true when the determiner "some" relates $A$ and $B$

• $\cap$: set intersection

• $\neq \emptyset$: the intersection is nonempty, so at least one element is in both sets

• "No dog barks": the set of dogs and the set of barkers are disjoint.

where:

• $A$: the restrictor set (e.g., the set of dogs)

• $B$: the scope set (e.g., the set of barkers)

• $\mathrm{no}(A)(B)$: true when the determiner "no" relates $A$ and $B$

• $\cap$: set intersection

• $\emptyset$: the empty set, so no element is in both $A$ and $B$

• "Most dogs bark": in this finite-set model, more than half of the dogs are in the set of barkers. With no dogs, the strict majority condition is false.

where:

• $A$: the restrictor set (e.g., the set of dogs)
• $B$: the scope set (e.g., the set of barkers)
• $\mathrm{most}(A)(B)$: true when the determiner "most" relates $A$ and $B$
• $\{x \in A : x \in B\}$: the set of elements of $A$ that are also in $B$
• $|\cdot|$: set cardinality (number of elements)
• $> |A|/2$: more than half the elements of $A$ are in $B$

These definitions support entailments. "No dogs bark" entails "No poodles bark" because the set of poodles is a subset of the set of dogs. If no dogs are in the barking set, no poodles are either. "Every dog barks" does entail "Every poodle barks" if poodles are dogs: if every dog barks, and every poodle is a dog, then every poodle barks. The direction depends on monotonicity, which we turn to next.

Monotonicity describes how entailments behave when you replace a constituent with a broader or narrower term.

• Upward entailing contexts preserve entailments from a narrower term to a broader term. "A poodle barked" entails "A dog barked." If you replace "poodle" with the broader "dog," the sentence remains true.
• Downward entailing contexts reverse the direction. "No dog barked" entails "No poodle barked." Here you replace "dog" with the narrower "poodle," and the entailment still holds.

Negation creates downward entailing contexts. "The door is not open" entails "The door is not ajar" if "ajar" is a way of being open. The broader term "open" is replaced by the narrower "ajar" under negation, and the entailment still holds precisely because negation reverses the direction of inference.

Monotonicity appears in analyses of negative polarity items such as "any," but it is a partial generalization rather than a complete licensing rule. Broader accounts also address contexts beyond downward entailment. Free-choice "any," as in "Any student may apply," needs to be distinguished from the negative-polarity use. For NLI, monotonicity gives a narrower, direct benefit: once a context's entailment direction and lexical inclusion relation are specified, they license controlled replacement pairs.

Quantifier scope is a major source of NLI difficulty. Consider:

Every student read a book.

This has at least two readings:

1. Every student read some book or other, possibly different books.
2. There is a shared book that every student read, without requiring it to be the only such book.

The first reading is:

where:

• $x$: a variable ranging over the entity domain, restricted by $\mathrm{student}(x)$
• $y$: a variable ranging over the entity domain, restricted by $\mathrm{book}(y)$
• $\mathrm{student}(x)$: true if $x$ is a student
• $\mathrm{book}(y)$: true if $y$ is a book
• $\mathrm{read}(x,y)$: true if $x$ read $y$
• $\forall x$: universal quantifier over domain entities, with the implication restricting the assertion to students
• $\exists y$: existential quantifier over books
• $\to$: logical implication

The second is:

where:

• $y$: a variable ranging over the entity domain, restricted by $\mathrm{book}(y)$
• $x$: a variable ranging over the entity domain, restricted by $\mathrm{student}(x)$
• $\mathrm{book}(y)$: true if $y$ is a book
• $\mathrm{student}(x)$: true if $x$ is a student
• $\mathrm{read}(x,y)$: true if $x$ read $y$
• $\exists y$: existential quantifier selecting a shared book witness $y$, without asserting uniqueness
• $\forall x$: universal quantifier over domain entities, with the implication restricting the assertion to students
• $\land$: logical conjunction

Now consider the hypothesis:

There is a book that every student read.

Under the first reading of the premise, the hypothesis is not entailed. The students might each have read a different book. Under the second reading, it is entailed directly. An evaluation that supplies one label for this pair must fix the intended scope reading. A robust NLI system needs to represent scope, or at least be sensitive to it, because otherwise it cannot tell which reading of the premise it is working with.

Negation scope is equally important. "All that glitters is not gold" is famously ambiguous between "Not everything that glitters is gold" and "Everything that glitters is not gold." The proverb usually means the first, but the surface order suggests the second. Models that ignore scope will mislabel such cases. Scope is not a decorative flourish; it changes the truth conditions, and therefore it changes the entailment relations.

Comparatives and temporal relations have their own entailment patterns.

• "Alice is taller than Bob" entails "Bob is shorter than Alice."
• "Alice is taller than Bob and Bob is taller than Carol" entails "Alice is taller than Carol" by transitivity.
• "Alice left before Bob arrived" entails "Bob arrived after Alice left."
• "Alice left after Bob arrived" entails "Bob arrived before Alice left," though tense and aspect can complicate temporal inference in other cases.

Temporal inference is especially tricky because it depends on event boundaries, aspect, and world knowledge. "Alice was reading when Bob arrived" entails that Alice was reading at the time Bob arrived. "Alice read a book when Bob arrived" is compatible with Alice starting to read after Bob arrived. The difference between progressive and simple past matters. A model that treats every "when" clause the same way will conflate two quite different temporal structures.

Not all phrase meaning follows from combining the literal word senses. Idioms like "kick the bucket" mean "die," but the individual words do not compose to that meaning in the usual way. "Spill the beans" means "reveal a secret." An analysis that combines only the literal word senses misses the idiomatic reading; the phrase-level interpretation must be represented somehow.

Metaphor and sarcasm require contextual interpretation. "That meeting was a roller coaster" need not describe an amusement park ride, and "Great, another delay" can convey frustration. A model that combines only literal word senses misses these readings. Compositionality itself does not impose that limitation: a meaning function can incorporate context or treat a conventional phrase as a lexical unit. Interpreting the utterance also draws on convention and world knowledge.

The modern NLI task is usually presented as a supervised classification problem. Each example has a premise, a hypothesis, and a label. The model outputs a probability distribution over entailment, contradiction, and neutral.

The task is not symmetric. The premise is the evidence; the hypothesis is the claim. Swapping them changes the inference. This is why NLI models are trained on ordered pairs, not unordered sets of sentences. It is also why the evaluation has to respect the order when it reports per-direction results.

The Recognizing Textual Entailment challenges established an application-oriented benchmark in the mid-2000s. RTE-1 and RTE-2 used binary entailment versus non-entailment judgments on small, hand-labeled text-hypothesis datasets. Their goal was to capture inferences needed across NLP applications, rather than to isolate controlled templates in the later HANS style. The RTE-2 organizers' announcement states the applied inference task and its relation to those applications.

The Stanford Natural Language Inference dataset, SNLI, enabled a large-scale supervised phase of NLI research. Bowman and colleagues introduced it in their 2015 paper. SNLI contains about 570,000 human-written premise-hypothesis pairs before filtering out cases without a consensus label. The premises are image captions, and crowdworkers wrote hypotheses. SNLI uses the entailment, contradiction, and neutral labels used throughout this chapter.

MultiNLI, described by Williams and colleagues in their 2018 paper, expanded the task to multiple genres. It contains about 433,000 pairs across ten genres, including fiction, government documents, travel guides, and telephone conversations. MultiNLI has matched and mismatched development/test splits. Five genres occur in its supervised training split; all ten occur in development and test. Mismatched evaluation therefore tests transfer to genres absent from that supervised split, without establishing that a pretrained encoder has never encountered those genres.

XNLI, described by Conneau and colleagues in their 2018 paper, uses newly collected English premise-hypothesis pairs translated into 14 additional languages, for 15 languages including English. It is a standard benchmark for cross-lingual transfer. The translations introduce their own artifacts, and negation and word order can differ substantially across languages, so a model that does well on English MNLI is not guaranteed to do well on XNLI.

Later datasets targeted specific weaknesses:

• ANLI (Adversarial NLI) used a human-and-model-in-the-loop process. Annotators wrote examples that fooled current models, then models were retrained, then new examples were written. The resulting benchmark tests models against those adversarial examples.
• HANS (Heuristic Analysis for NLI Systems) constructs controlled cases where lexical-overlap, subsequence, and constituent heuristics succeed or fail. It is intended to diagnose those heuristics, not to represent the full distribution of inference in text.
• MedNLI tests clinical inference grounded in patient histories. SciTail derives hypotheses from school-science questions and answer choices, with retrieved web sentences as premises.
• DocNLI uses document-level premises and variable-length hypotheses, extending the input beyond isolated sentence pairs.

These datasets do not all share the same label alphabet. Binary non-entailment, for example, does not preserve a separate contradiction category:

Different datasets address scale, genre coverage, language transfer, or targeted failure diagnosis. Select evaluations for the capabilities needed in the application rather than treating their scores as interchangeable.

NLI datasets are not neutral reflections of semantic competence. They contain artifacts from how they were constructed.

A useful artifact check is the hypothesis-only baseline. If you train a classifier on the hypothesis alone, ignoring the premise, you can achieve accuracy well above chance on SNLI and other datasets. The hypotheses were written by crowdworkers who tended to use certain words and constructions for certain labels. For example, hypotheses containing "not" or "nobody" may be more likely to be contradictions. A model can learn these label-associated patterns without learning anything about the premise.

This does not mean SNLI is useless. It means that high accuracy on SNLI is not proof of semantic understanding. It also means that evaluation must include controls for hypothesis-only information, lexical overlap, and other shortcuts. The right response is not to discard the dataset but to interpret its scores with appropriate skepticism.

Which label is hardest depends on the model and evaluation set. Neutral includes varied cases, so inspect per-class errors rather than assuming a fixed difficulty order. Predicting neutral more often can raise its recall while increasing false-neutral errors; predicting it on every example would give perfect neutral recall when gold neutral examples are present, without establishing any inference ability. Model uncertainty is separate from the semantic neutral relation and may call for abstention instead.

Class imbalance varies by dataset and split. Always check the label distribution before reporting accuracy. A model that predicts the majority label can look better than it is. Macro-F1 gives each label equal weight in the aggregate score. Per-class recall reports a separate value for each label, so minority class recall remains visible.

NLI models have evolved from feature-based classifiers to large pretrained transformers. The core problem remains the same: represent the relationship between two sentences.

Earlier RTE systems often combined hand-engineered features. For example, MENT's RTE-2 system used syntactic and thesaurus features with an alignment score. A feature-based baseline can combine:

• Lexical overlap between premise and hypothesis.
• Word alignment and edit distance.
• Syntactic parse features.
• Named entity overlap.
• Negation detection.
• A text-similarity feature, building on TF-IDF or BM25. Such a score measures a form of similarity or relevance, not entailment by itself.

A logistic regression or SVM classifier could combine these features. Their limitations depend on the chosen features: a word-count representation omits order, while explicit syntax or negation features can preserve some distinctions. A classifier that succeeds through lexical overlap still needs tests where overlap and entailment disagree.

Two common designs illustrate different ways to represent the sentence pair.

A bi-encoder encodes the premise and hypothesis separately:

where:

• $p$: the premise sentence
• $h$: the hypothesis sentence
• $f_\theta$: the encoder function parameterized by weights $\theta$
• $u$: the encoded representation (vector) of the premise
• $v$: the encoded representation (vector) of the hypothesis

It then combines the two vectors, for example:

Here $u,v \in \mathbb{R}^{d}$ have the same dimension. Semicolons concatenate vectors, $|u-v|$ takes the componentwise absolute difference, and $\odot$ multiplies corresponding components. The pair vector $z \in \mathbb{R}^{4d}$ has four blocks of $d$ entries. Its ordered $u$ and $v$ blocks can preserve direction even though the difference and product blocks are symmetric.

A classifier uses the combined vector $z$. Separate encoding allows sentence vectors to be cached and reused across pairs. It omits direct cross-sentence token interaction during encoding. A sentence encoder can still represent internal negation and role information, but the pair classifier must recover the relation from the fixed vectors.

A cross-encoder concatenates the premise and hypothesis into a single input:

where:

• $[\text{CLS}]$: the special classification token whose final hidden state is used for prediction
• $p$: the token sequence of the premise
• $[\text{SEP}]$: the special separator token marking segment boundaries
• $h$: the token sequence of the hypothesis

A transformer encodes the whole sequence. With an unblocked bidirectional attention mask, self-attention permits each premise token to attend to each hypothesis token. The final hidden state at the [CLS] position, or a pooled representation, is passed to a linear classification head:

where:

• $p_\theta(y \mid p,h)$: the predicted probability of label $y$ given premise $p$ and hypothesis $h$
• $y$: a label in {entailment, contradiction, neutral}
• $h_{[\text{CLS}]}$: the final hidden state at the [CLS] position
• $W$: the weight matrix of the linear classification head
• $b$: the bias vector of the classification head
• $\operatorname{softmax}(\cdot)_y$: the softmax function evaluated at the entry for label $y$

With a pooled hidden state $h_{[\mathrm{CLS}]} \in \mathbb{R}^{d}$, a three-label head uses $W \in \mathbb{R}^{3 \times d}$ and $b \in \mathbb{R}^{3}$. Its output has one logit per label. This hidden-state symbol is distinct from the hypothesis sentence $h$.

A cross-encoder requires joint encoding of each pair, whereas a bi-encoder can reuse an already computed sentence vector. Joint encoding permits token-level interaction across the pair; the resulting accuracy depends on the model, training data, and evaluation set. For a workload comparing many new pairs, this joint computation cannot use a cache of independently encoded sentences in the same way. Compute comparisons still depend on sequence length, architecture, batching, and hardware.

The special tokens [CLS] and [SEP] were introduced in Special Tokens. Their role here is structural: [CLS] aggregates the pair representation, and [SEP] marks the boundary between premise and hypothesis.

For this supervised classifier, take a nonempty dataset of $N>0$ labeled pairs with one gold label per pair. Minimize the mean cross-entropy loss:

where:

• $\mathcal{L}(\theta)$: the cross-entropy loss for model parameters $\theta$
• $N$: the number of labeled pairs in the dataset
• $i$: an index over training pairs
• $p_i$: the premise of the $i$-th pair
• $h_i$: the hypothesis of the $i$-th pair
• $y_i$: the gold label of the $i$-th pair
• $p_\theta(y_i \mid p_i, h_i)$: the model's predicted probability of the gold label
• $\log$: the natural logarithm

Minimizing this objective rewards higher probability on the supplied gold labels. One training recipe is to fine-tune a pretrained transformer on MNLI, then adapt it to a domain-specific dataset. This is a recipe to evaluate, not a guarantee of transfer.

Class weighting, label smoothing, and adversarial training are optional refinements. Their suitability needs separate evaluation; changing training weights, targets, or examples does not by itself establish semantic generalization. The objective teaches the model to match labels, and if the labels are predictable from shortcuts, the model will happily learn the shortcuts.

Large language models can perform NLI zero-shot or few-shot. A typical prompt asks:

Premise: ...

Hypothesis: ...

Does the premise entail the hypothesis, contradict it, or neither? Answer with one of: entailment, contradiction, neutral.

There are two ways to score the answer:

1. Generate the label as text and parse it.
2. Score the probability of each label token or verbalizer and choose the highest.

Free generation introduces decoding and parsing choices. Temperature zero makes token selection greedy, but does not by itself guarantee reproducible inference across hardware, software, or batching configurations. Prompt wording, demonstration choice, and demonstration order can also change predictions. Scoring label tokens fixes the candidate outputs, but still depends on the verbalizer and tokenization. For example, "entailment" may tokenize into multiple pieces, so score the sequence, not just one token.

Prompting an LLM provides a way to attempt the task without task-specific training examples. It does not establish an advantage over a fine-tuned cross-encoder. Compare model families on the same evaluation protocol and inspect their errors.

NLI evaluation can test controlled semantic distinctions alongside transfer across genres, domains, and languages. Agreement with one benchmark's labels answers a narrower question than success across all those settings.

In-distribution test sets share the same annotation process and genre distribution as training data. They are useful, but they can overestimate generalization. A model can score very high in-distribution while relying on cues that vanish the moment the data changes.

Out-of-distribution evaluation uses a different genre, domain, or annotation process. MultiNLI's mismatched split is an early example. Domain-specific datasets like MedNLI test whether a model can handle medical text. Cross-lingual evaluation with XNLI tests transfer across languages.

Adversarial evaluation is more targeted. ANLI uses human-written examples designed to fool models. HANS uses controlled templates to test specific heuristics. Contrast sets make small, meaningful perturbations to examples, typically changing the gold label. These sets are not necessarily representative of natural text; they test particular local distinctions. Failure on a controlled case can identify a distinction to investigate even when aggregate accuracy is high.

The hypothesis-only baseline is the simplest artifact check. Train a model on the hypothesis alone. If it performs well above chance, the dataset contains label-associated patterns in the hypotheses. This does not invalidate the dataset, but it means that premise-aware models may be learning some of the same shortcuts.

Lexical overlap is another common shortcut. A model may predict entailment whenever the hypothesis words appear in the premise. HANS tests this with examples like:

• Premise: "The lawyer questioned the witness."
• Hypothesis: "The witness questioned the lawyer."

The words overlap heavily, but the relation is neutral. A model that relies on overlap may predict entailment. A model that represents argument order and thematic roles should predict neutral.

HANS distinguishes three heuristics:

• Lexical overlap: if all hypothesis words appear in the premise, predict entailment.
• Subsequence: if the complete hypothesis is a contiguous subsequence of the premise, predict entailment. This is the definition used in HANS Table 1, rather than the broader mathematical use of subsequence.
• Constituent: if the complete hypothesis corresponds to a syntactic constituent of the premise, predict entailment.

A heuristic can agree with the gold labels on many examples without being a valid inference rule. Cases where the heuristic and gold label diverge test whether that shortcut is sufficient. Passing those cases still requires follow-up checks; a model may use another shortcut.

A good NLI evaluation battery covers the phenomena that make sentence meaning hard. The following categories are a useful starting point.

• Negation: "The door is open" versus "The door is not open." Test whether predictions respond to its scope rather than only its presence.
• Numerals and quantities: "At least three dogs" versus "Exactly three dogs." "More than five" versus "Fewer than five." Numerical inference is compositional and easy to perturb.
• Entity swapping: "Alice thanked Bob" versus "Bob thanked Alice." High lexical overlap, different roles.
• Quantifier scope: "Every student read a book" versus "There is a book that every student read." The first does not entail the second under the distributive reading.
• Temporal relations: "Alice left before Bob arrived" versus "Bob arrived before Alice left." Order matters.
• World knowledge: "The Eiffel Tower is in Paris" versus "The Eiffel Tower is in France." The inference requires geographical knowledge.
• Presupposition: "John stopped smoking" versus "John used to smoke." On the intended presuppositional reading, prior smoking remains backgrounded under negation; test that reading explicitly.
• Lexical ambiguity: "The bank raised rates" versus "The river bank is steep." Same word, different senses.
• Collective versus distributive: on the collective reading, "Alice and Bob lifted the piano" does not entail "Alice lifted the piano alone."
• Long premises: A relevant sentence buried in a long document. Test whether adding irrelevant context changes the relation prediction.

These categories overlap: one example can involve negation, scope, and world knowledge. Report category-level results alongside the aggregate score so that a strong category does not conceal a failure on an important one. Deployment decisions also need evidence from the intended application.

For models trained on public web text, overlap with publicly released evaluation examples is a possibility that requires investigation. A benchmark score alone cannot establish or rule out memorization. The GPT-3 study examined training/test overlap; whether a particular NLI dataset contaminated a particular model still requires evidence about those inputs.

Contamination is hard to prove or rule out. Practical mitigations include:

• Evaluate on a private or recently collected dataset.
• Check n-gram overlap between the evaluation set and known pretraining corpora, if available.
• Use time-split evaluation, where the test set was created after the model's training cutoff.
• Report multiple benchmarks, including adversarial and domain-specific sets.
• Treat unusually high zero-shot scores with suspicion.

Prompt sensitivity also matters for LLM-based NLI. Test whether predictions are stable under prompt changes rather than relying on one phrasing. An evaluation can:

• Test multiple prompt templates.
• Permute the order of label options.
• Report variance across prompts.
• Use constrained scoring over label tokens rather than free generation when possible.
• Report decoding settings, including temperature, top-p, and random seeds.

Even greedy decoding is not a guarantee of determinism across hardware and batch sizes. If reproducibility matters, fix the environment and report it. A reader who cannot reproduce your setup cannot check your conclusions.

The following measurements answer different evaluation questions:

• Accuracy: overall correctness.
• Macro-F1: averages per-class F1, giving each label equal weight.
• Per-class precision and recall: report all labels; the weakest class depends on the model and evaluation set.
• Calibration: whether predicted probabilities match empirical correctness. A model that is 90% accurate but always outputs 99% confidence is badly calibrated.
• Selective accuracy: accuracy on examples where the model's confidence exceeds a threshold. This is useful when a system can defer to a human.
• Adversarial accuracy: performance on HANS, ANLI, or domain-specific stress tests.
• Confidence intervals: resample independent evaluation units and recompute the metric. Cluster related pairs by source document or template when needed. State the population and resampling scheme; an interval conditional on a fitted model does not include all training, annotation, or distribution-shift uncertainty.
• Paired comparisons: for accuracy, McNemar's test uses paired correct/incorrect outcomes. For macro-F1 or another aggregate metric, use paired resampling or randomization appropriate to that metric and sampling design, applying the same sampled units to both models.
• Human agreement: annotator agreement on the evaluation set, especially for neutral versus entailment.

A small test set can produce large confidence intervals. Whether a difference of one or two examples is informative depends on the sample size, pairing, and uncertainty; the count alone is insufficient. An interval describes uncertainty under its stated sampling assumptions. It cannot make a hand-picked diagnostic battery representative. Test choice also depends on the metric and data dependence.

If you are building or deploying an NLI system, a practical protocol looks like this:

1. Define the label semantics for your domain. Is a presupposition an entailment? Is a plausible inference enough, or do you need strict logical consequence?
2. Check the label distribution in your training and test data. Report majority-class baselines.
3. Run a hypothesis-only baseline to detect annotation artifacts.
4. Build a diagnostic battery covering negation, numerals, entity swaps, quantifiers, temporal relations, and presupposition.
5. Test lexical overlap shortcuts with controlled pairs.
6. Evaluate out-of-distribution on a different genre or domain.
7. Check for contamination if using a public benchmark or an LLM.
8. Test prompt sensitivity if using an LLM.
9. Report per-category results and confidence intervals.
10. Do an error analysis on a random sample and on the worst-performing category.

This protocol will not make NLI solved. It will make your claims more honest.

We will now build a small NLI battery and run an explicit lexical baseline. This deterministic example uses Python, NumPy, pandas, and Matplotlib from the locked book environment; it downloads no models or data. The baseline intentionally ignores word order and lexical hierarchies so that the battery exposes its limitations.

The battery is intentionally small. It is a teaching tool, not a benchmark. The goal is to inspect controlled failures and practice computing evaluation metrics. The manually chosen score weights are not learned or tuned on a held-out split. Softmax normalizes those scores to sum to one; it does not calibrate them as empirical confidence estimates.

The hidden setup at the start of the chapter seeds the book's plotting utilities once. Each figure applies the shared book style when it renders.

Next, import the small numerical and plotting dependencies. The tokenization below lowercases text and extracts ASCII letter/digit runs; it is deliberately limited to these English teaching examples.

The baseline uses hypothesis-word overlap, a mismatch in negation words, and a small list of contrasting terms. These are fallible cues: opposite words need not denote incompatible events, and a negation word need not have the same scope in both sentences. The coefficients below are fixed illustrative choices, so the scores describe this heuristic rather than a trained NLI system.

The output columns always use LABELS in the declared order: contradiction, entailment, neutral. There is no external model configuration or environment-dependent fallback.

The batch function checks that every premise has a hypothesis. It returns a normalized heuristic score for each label and selects the largest score. Exact ties follow the declared label order because the first maximum is selected. An empty batch returns an empty list.

The battery covers simple entailment, contradiction, neutral pairs, lexical ambiguity, event structure, and downward monotonicity. Gold labels use the intended readings stated here. Hold referents and relevant times fixed when descriptions are intended to corefer. In the bank example, the premise uses the financial sense and the hypothesis the geographical sense, so they concern different entities. The train-arrival pair refers to one arrival, and the meeting-time pair to one scheduled start; different events would change the judgment. The child-and-park example uses an outdoor park reading, and the museum pair uses one recurring weekly schedule with a binary daily open/closed status. The toy battery has no training/test split because no parameters are fitted.

The first probe isolates lexical specificity: a poodle barking entails a dog barking, while the reverse is neutral. The second probe assumes the same restaurant, time, and a binary open/closed status. Under that explicit convention, "open" and "not closed" are equivalent. The lexical baseline is not expected to implement either semantic rule correctly.

Presuppositions often project under negation in the readings considered here. A model that only learns surface entailment may treat the negated premise differently from the positive one, even though the presupposition survives in both.

The three-way label set does not separately identify presupposition. If both positive and negated premises receive entailment labels, the outputs alone do not establish which cues or meaning representations produced them. Additional controlled probes are needed to investigate the mechanism.

The stress battery targets lexical overlap, negation, and downward monotonicity. The last two examples are a minimal pair: "No dogs are allowed" entails "No puppies are allowed," but "No puppies are allowed" does not entail "No dogs are allowed."

The lawyer-witness example is the classic lexical overlap trap. The premise and hypothesis share almost all content words, but the grammatical roles are reversed. A model that attends to argument order should predict neutral. A model that counts overlapping words may predict entailment. The minimal pair about dogs and puppies tests the other end of the same idea: downward monotonicity means the inference runs from the broader term to the narrower one, not the other way around.

We can compute a confusion matrix and macro-F1 on the toy battery. With only twelve hand-chosen examples, these numbers describe this battery; they are not a representative estimate of population performance. The point is to practice reporting per-class behavior, not to rank models.

With twelve examples, the confusion matrix is sparse. That is normal for a teaching battery. For an application evaluation, choose a sampling design and sample size that fit the target population and required uncertainty, then report category-level results with an appropriate uncertainty analysis.

NLI evaluates a relation under a specified labeling policy; it is not a complete theory of meaning. The three-label format does not identify every meaning layer. A policy may count a projected presupposition as entailment, or reserve entailment for asserted content. It must also state how to treat conversational implicature. Neutral is a residual category that mixes underspecification, missing world knowledge, ambiguity, and pragmatic effects. A model can achieve high aggregate accuracy while systematically failing on negation, quantifier scope, or entity swapping.

The datasets themselves carry artifacts. SNLI and MultiNLI were built with crowdworkers, and crowdworkers are sensitive to the hypotheses they are writing. Hypothesis-only baselines perform well above chance. Lexical overlap is a strong cue. HANS targets specified syntactic heuristics, while ANLI collects examples adversarially against models. A high MNLI score is evidence that a model has learned something useful, but it is not evidence that the model has solved sentence meaning.

Large language models add new evaluation problems. Public NLI examples may occur in pretraining corpora; establishing contamination for a particular model requires evidence about that model and its data. Prompt wording and demonstration order can change predictions. Greedy decoding does not by itself guarantee identical results across hardware or batching configurations. A responsible evaluation reports multiple prompts, checks for contamination, and includes adversarial and domain-specific tests. Calibration and selective accuracy measure probability reliability and accuracy at a stated coverage; neither establishes that the model represents its own lack of knowledge.

NLI models are also used as zero-shot classifiers. The source-consistency, evidence-reranking, and factual-consistency methods introduced earlier provide other concrete applications. They can help decide whether a retrieved passage supports a generated claim, whether a summary is entailed by a source document, or whether two product descriptions contradict each other. But they should not be treated as truth arbiters. The premise is assumed true. The model checks the relation, not the facts. A system that forgets this distinction can confidently endorse a premise that is itself false, or reject a hypothesis that happens to be true but is not supported by the given text.

The most important lesson is methodological. Evaluate NLI systems on the phenomena you care about, not just on a leaderboard. Build controlled pairs. Test negation and scope. Test presupposition. Test lexical overlap. Test domain shift. Report per-category results and uncertainty. Use human review for high-stakes decisions. The next chapters extend this semantic toolkit to semantic roles, event extraction, entity linking, and semantic parsing. Those tasks will reuse the same core ideas: composition, scope, lexical entailment, and the difference between what a sentence says and what a speaker means.

Sentence meaning is compositional and truth-conditional. It depends on word meanings, syntactic structure, scope, and context. NLI formalizes three core relations: entailment, contradiction, and neutrality. Entailment is directional. Contradiction is not the same as falsehood. Neutral is not ignorance; it means the premise does not determine the hypothesis.

Presupposition and conversational implicature differ from ordinary asserted entailment. Presuppositions can project under negation, depending on context; conversational implicatures are typically cancellable. A three-way NLI label does not separately identify these meaning layers.

Transformer cross-encoders predict labels from paired text, with datasets such as SNLI, MultiNLI, and ANLI supplying supervision and evaluation examples. They can still learn shortcuts. Evaluate those risks with diagnostic cases, hypothesis-only baselines, category-level results, and checks appropriate to the data and model. For prompted LLMs, record the prompt and decoding protocol and investigate benchmark overlap.

The code battery in this chapter runs a fixed lexical baseline on controlled examples, tests directional entailment, probes presupposition, and computes a confusion matrix. Its hand-selected weights and normalized scores are illustrative; they are not measurements of a pretrained model. The results are illustrative, not definitive. The main takeaway is that NLI is a way to study sentence meaning, not a replacement for semantic theory. In the next chapter, we move from sentence-level entailment to the internal structure of predicates and their arguments: semantic roles and argument structure.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about sentence meaning and natural language inference.