How Language Models Work (in Plain English) › What the Model Knows (and Doesn't Know)

This prediction mechanism gives language models some impressive capabilities, but also important limitations.

What models are good at:

• Language fluency: They generate grammatically correct, natural-sounding text because they've seen millions of examples of good writing
• Factual recall: They can answer many factual questions because facts appeared repeatedly in their training data ("Paris is the capital of France" shows up in countless documents)
• Pattern recognition: They can complete common patterns, solve typical problems, and follow familiar formats
• Context understanding: They track what's been said earlier in a conversation and maintain coherence

What models struggle with:

• Exact facts: If something appeared rarely in training data, the model might not know it or might confuse it with similar information
• Current events: Models are trained on data up to a certain cutoff date. They don't know what happened after that (unless we give them tools to look things up; more on that in Chapter 5)
• Precise calculations: While they can do simple math, complex calculations aren't their strength because they're predicting tokens rather than executing arithmetic operations
• True reasoning: They can follow reasoning patterns they've seen before, but they don't "think" in the way humans do. They're very sophisticated pattern matchers

This is why our AI agent will eventually need more than just a language model. We'll add tools for calculations, memory for context, and structured reasoning approaches to overcome these limitations. But the language model is the foundation. It's what lets our agent understand language and generate responses.