Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Scaling Up without Breaking the Bank › Optimizing Prompts to Reduce Costs

Michael BrenndoerferAugust 28, 20253 min read

AI Agent HandbookAll chapters

Part 15 of 15Performance and Cost Optimization - Doing More with Less

Chapter 44Scaling Up without Breaking the Bank › Optimizing Prompts to Reduce Costs

Reading section 7 of 13 · Optimizing Prompts to Reduce Costs · 3 minSection 7 of 13 · 3 min

Section 7 of 13 · 3 min read

Scaling Up without Breaking the Bank › Optimizing Prompts to Reduce CostsOne section · Sections in this chapter7 / 13 ↓

Sections in this chapter13

Here's a scaling secret: small optimizations in your prompts can lead to massive cost savings at scale.

Consider this: if you reduce the average response length from 500 tokens to 400 tokens, you save 20% on output costs. For a system handling 100,000 requests per month, that 20% could mean hundreds or thousands of dollars in savings.

Let's look at some optimization strategies:

Be specific about length: Instead of letting the model generate however much text it wants, specify the desired length.

In[2]:

Code

## Before: Uncontrolled length
prompt = "Explain how photosynthesis works."

## After: Controlled length
prompt = "Explain how photosynthesis works in 3-4 sentences."

The second prompt typically generates 50-100 tokens instead of 200-300. That's a 60-70% reduction in output costs.

Use structured output formats: When you need specific information, ask for it in a structured format rather than prose.

In[4]:

Code

## Before: Prose response
prompt = "Tell me about the weather in Boston today."

## After: Structured response
prompt = """What's the weather in Boston today? 
Respond in this format:
Temperature: [value]
Conditions: [description]
Precipitation: [yes/no]"""

The structured format produces shorter, more predictable responses. You also get the benefit of easier parsing.

Reuse context when possible: If you're making multiple requests about the same topic, consider batching them into a single request.

In[6]:

Code

## Before: Three separate requests
queries = [
    "What's the capital of France?",
    "What's the population of France?",
    "What's the currency of France?",
]

## After: One batched request
query = """Answer these questions about France:

1. What's the capital?
2. What's the population?
3. What's the currency?

Keep each answer to one sentence."""

This reduces the number of API calls and eliminates redundant context processing. You pay for input tokens once instead of three times.

Cache common responses: For frequently asked questions, cache the responses instead of calling the API every time.

In[8]:

Code

## Using Claude Sonnet 4.5 for agent operations
import hashlib


class CachedAgent:
    def __init__(self, client):
        self.client = client
        self.cache = {}

    def query(self, prompt, use_cache=True):
        """Query with optional caching"""
        # Create a cache key from the prompt
        cache_key = hashlib.md5(prompt.encode()).hexdigest()

        # Check cache
        if use_cache and cache_key in self.cache:
            print("Cache hit!")
            return self.cache[cache_key]

        # Call API
        response = self.client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            messages=[{"role": "user", "content": prompt}],
        )

        result = response.content[0].text

        # Store in cache
        if use_cache:
            self.cache[cache_key] = result

        return result


## Use cached agent
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
agent = CachedAgent(client)

## First call: API request
result1 = agent.query("What's the capital of France?")

## Second call: cached (no API cost)
result2 = agent.query("What's the capital of France?")

For questions that many users ask, caching can reduce API costs by 50% or more.