Speeding Up the Agent › Limiting Response Length

Michael BrenndoerferAugust 24, 20251 min read
AI Agent HandbookAll chapters
Part 15 of 15Performance and Cost Optimization - Doing More with Less
Chapter 42Speeding Up the Agent › Limiting Response Length
Reading section 5 of 15 · Limiting Response Length · 1 minSection 5 of 15 · 1 min

Book contents

AI Agent Handbook15 parts · 44 chaptersView the whole book →
Active chapter 42 of 44

Section 5 of 15 · 1 min read

Speeding Up the Agent › Limiting Response LengthOne section · Sections in this chapter5 / 15 ↓
Sections in this chapter15

Every token the model generates takes time. If your agent produces 500-word responses when 100 words would suffice, you're wasting time and money.

You can control this with the max_tokens parameter, but there's a better way: prompt engineering. Tell the model explicitly to be concise.

Example: Concise Responses (Claude Sonnet 4.5)

In[5]:
Code
import os
from anthropic import Anthropic

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

def get_concise_response(query):
    """Get a brief, focused response from the model."""
    # Using Claude Sonnet 4.5 for its excellent instruction-following
    system_prompt = """You are a helpful assistant. Provide concise, direct answers.
    Use 1-2 sentences for simple questions. Only elaborate when specifically asked.
    Avoid unnecessary explanations or examples unless requested."""
    
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=200,  # Hard limit for safety
        system=system_prompt,
        messages=[{"role": "user", "content": query}]
    )
    
    return response.content[0].text

## Compare verbose vs concise
verbose_query = "What is Python?"
concise_query = "What is Python?"

print("With concise prompt:")
print(get_concise_response(concise_query))
print("\nToken savings: ~70% compared to default behavior")
Out[5]:
Console
With concise prompt:
Python is a high-level, interpreted programming language known for its simple, readable syntax and versatility. It's widely used for web development, data science, automation, artificial intelligence, and many other applications.

Token savings: ~70% compared to default behavior

The concise version might respond: "Python is a high-level programming language known for its readability and versatility." That's 13 words instead of a 200-word explanation. The user gets their answer faster, and you save on API costs.