Speeding Up the Agent › Why Speed Matters

Let's start with a scenario. Your assistant is deployed, and a user asks: "What's 47 times 83?"

The agent springs into action. It sends the query to Claude Sonnet 4.5, which thinks about the problem, decides to use the calculator tool, performs the calculation, and generates a response. Total time: 4.2 seconds.

Now imagine the user asks ten questions in a row. That's 42 seconds of waiting. The user gets impatient. They start to wonder if something's broken. They might even give up and use a different tool.

Speed isn't just about user experience, though that's important. It's also about cost. Most language model APIs charge per token generated. A slower agent that generates verbose responses costs more to run. If your agent takes twice as long and generates twice as many tokens, you're paying roughly four times as much per interaction.

The challenge is balancing speed with capability. You want your agent to be fast, but not at the expense of accuracy or usefulness. The techniques in this chapter help you find that balance.