Classification Applications › Intent Detection

Intent detection is the problem of identifying what a user wants to accomplish from a natural language utterance. It is the classification task that powers conversational AI: when a user says "What's the weather like tomorrow?", intent detection identifies the WeatherQuery intent. When they say "Cancel my subscription," it identifies the CancelSubscription intent. The detected intent then drives downstream action: querying a weather API, opening an account management flow, triggering a confirmation dialog.

Intent detection is structurally identical to topic classification, but the characteristics of the task are quite different. Where topic classification deals with documents ranging from sentences to pages, intent detection operates on short utterances, typically under 20 words. Where topic categories are broad thematic domains, intents are narrow, action-oriented categories defined by the application's capability set. And where topic classifiers deal with a relatively stable label set, intent detection systems must handle a fundamental tension between the fixed set of supported intents and the unbounded diversity of user expression.

The intent schema is the enumeration of all actions the system can take. For a banking assistant, it might include: CheckBalance, TransferFunds, PayBill, ReportFraudulentCharge, CloseAccount, OpenAccount. The number of intents varies widely across applications: voice assistants may support thousands, domain-specific bots may support fewer than 20.

Designing the intent schema well requires thinking carefully about the granularity of actions. If you define intents too broadly, the downstream action logic has insufficient information to proceed. "AccountManagement" as a single intent gives no guidance about what the user wants to do with their account. If you define intents too narrowly, the number of categories explodes and each category has too few training examples. A useful rule of thumb is that each intent should correspond to a distinct action that the system takes, and that action should differ in its effect from the actions triggered by adjacent intents.

A critical consideration that separates intent detection from generic topic classification is out-of-scope (OOS) detection: identifying when a user's intent falls outside the supported capability set. A user asking "Can you recommend a good restaurant?" to a banking assistant is expressing a valid intent, but one the system cannot fulfill. A naive intent classifier will assign this to the closest in-domain intent, perhaps ReportFraudulentCharge if the model picks up some spurious pattern, producing a confusing response. OOS detection adds either a special OOS class or a threshold-based rejection mechanism.

The threshold approach defines a confidence floor: if $\max_k P(l_k \mid d) < \tau$ for some threshold $\tau$, the system treats the utterance as out-of-scope. Choosing $\tau$ involves a tradeoff between false positives (in-scope utterances incorrectly rejected) and false negatives (OOS utterances incorrectly assigned to an in-domain intent). In practice, calibrating $\tau$ on a held-out OOS development set is essential: the default softmax probabilities from a fine-tuned classifier are often overconfident, assigning high probability to the nearest in-domain intent even for utterances far from the training distribution.

The plot below illustrates how the false positive rate (legitimate queries incorrectly rejected) and false negative rate (OOS queries incorrectly handled as in-scope) vary as you sweep the threshold from 0 to 1.

The optimal threshold depends on the cost structure of your application. If failing to catch an OOS query costs more than occasionally rejecting a legitimate one, as in a safety-critical system, you push the threshold up, accepting higher false positives to reduce false negatives. If annoying users with unnecessary rejections is your primary concern, you push it down. Neither end of the spectrum is universally correct: the threshold is a business decision as much as a modeling one.

An alternative to threshold-based OOS detection is to train an explicit OOS class using examples of out-of-scope utterances collected from real traffic or synthesized by paraphrasing unrelated tasks. The explicit class approach tends to work better when you have enough OOS examples to characterize the distribution, because the model learns what OOS looks like rather than simply looking for low confidence. The threshold approach is simpler and does not require OOS labels, making it the practical default for new deployments where OOS data is not yet available.

Short utterances present specific challenges for classification models. A tweet, a search query, or a voice command might be four words long. The signal-to-noise ratio in short text is low: every word matters, there is little context to resolve ambiguity, and the feature space is sparse.

Transformers handle short text better than earlier sequential models because self-attention allows the model to consider all pairwise interactions between tokens regardless of their distance. But very short sequences do not give the model much to work with. "Call her" is ambiguous: it could mean PhoneContact, ReadMessage, or something else entirely depending on context the utterance does not provide. Contextual intent detection systems that maintain conversation history address this by feeding prior turns into the model, allowing "Call her" to be resolved against "I need to reach my sister."

Another strategy for short-text classification is data augmentation: generating paraphrases of short training examples to expose the model to more surface variation. Large language models are effective paraphrase generators. Given "What's my balance?", a language model can generate "How much money do I have?", "Show me my account balance", "Can you check my balance?", and dozens of variants. Training on these augmented examples improves performance across lexical variations without requiring human annotation of the additional examples.

Intent detection rarely operates in isolation in production systems. Real conversational AI systems perform joint intent detection and entity extraction: the model simultaneously identifies the intent (BookFlight) and the slot values that parameterize the action (departure city: "New York", destination city: "London", date: "next Friday").

This joint modeling problem can be formulated as a multi-task classification and sequence labeling task. The model produces:

1. A distribution over intents from the [CLS] representation (classification)
2. A label for each token indicating whether it is part of a slot value, and which slot (sequence labeling, typically using BIO tagging)

Joint modeling generally outperforms pipeline systems (intent detection followed by separate slot filling) because the two tasks share information. The word "fly" in "I want to fly to Paris" is evidence for both the BookFlight intent and the existence of a destination slot. A joint model can exploit this co-occurrence, while a pipeline model must perform each step in isolation.

As we discussed in the sequence labeling chapter, BIO tagging assigns each token a label indicating whether it is the Beginning, Inside, or Outside of a named span. In the slot filling context, the tags carry additional information: B-DESTINATION, I-DESTINATION, B-DATE, O, etc.

The shared representation between intent and slot prediction creates a beneficial interaction. Consider the utterance "Book a flight to Paris on Friday." The word "Paris" provides strong evidence for the BookFlight intent (since booking flights involves destinations) and simultaneously is the value of the DESTINATION slot. In a pipeline system, the slot filler would only see "Paris" after the intent has been decided, missing the opportunity to use the slot evidence when making the intent prediction. In a joint model, the transformer attends over the full utterance simultaneously, allowing each prediction to inform the other. This mutual reinforcement is why joint models outperform pipelines, particularly on ambiguous utterances where neither the intent nor the slot values are obvious from a partial view of the text.

Training a joint model requires labeled data in a specific format: each utterance must carry both an intent label and token-level slot annotations. Public datasets like ATIS (Airline Travel Information System) and SNIPS provide this format. When you lack labeled data for a new domain, zero-shot and few-shot approaches based on generative models can be used to bootstrap the joint model, though performance is lower than fully supervised training.