How Language Models Work (in Plain English) › Different Sizes, Different Capabilities

Not all language models are created equal. They come in different sizes, usually measured by the number of parameters (those internal knobs we mentioned earlier).

Smaller models, with millions to a few billion parameters, are faster and cheaper to run. They work well for simple tasks like classification or generating short responses, but they may struggle with complex reasoning or rare knowledge. Think of them as quick-thinking but less experienced assistants.

Larger models, with tens to hundreds of billions of parameters, are slower and more expensive to run, but they are better at complex tasks that require tracking subtle distinctions or following detailed instructions. They're more likely to have learned rare facts or specialized knowledge because they've had the capacity to absorb more patterns during training. These are your deep-thinking, well-read experts.

This size-capability relationship isn't arbitrary. More parameters mean more capacity to encode context-dependent patterns. A small model might learn that "bank" relates to "money," but a larger model can distinguish between financial banks, river banks, and even the verb "to bank on something" based on subtle context clues. This additional capacity comes at a cost, though. A model with 100 billion parameters might take 10 times longer to generate a response than one with 10 billion parameters, and it requires significantly more memory and computational resources.

For our personal assistant agent, we'll likely use a large, capable model (like GPT-4, Claude, or Gemini) because we want it to handle a wide variety of tasks well. But knowing that smaller models exist is useful. Sometimes you can use a small model for simple subtasks to save time and cost, a strategy we'll explore in Chapter 15. This approach, called "model routing," lets you match the model size to the task complexity, getting the best balance of quality and efficiency.