Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Maintenance and Updates › Managing Costs Over Time

Michael BrenndoerferAugust 22, 20253 min read

AI Agent HandbookAll chapters

Part 14 of 15Deployment and Operations - From Prototype to Production

Chapter 41Maintenance and Updates › Managing Costs Over Time

Reading section 7 of 19 · Managing Costs Over Time · 3 minSection 7 of 19 · 3 min

Section 7 of 19 · 3 min read

Maintenance and Updates › Managing Costs Over TimeOne section · Sections in this chapter7 / 19 ↓

When you first deployed your agent, costs were probably low. A few dollars a month for API calls, maybe some hosting fees. But as usage grows, costs can grow too. Maintenance includes keeping costs under control.

Track Where Money GoesLink Copied

Start by understanding your cost breakdown. Most cloud platforms and API providers have billing dashboards showing:

Language model API calls (usually the biggest cost)
Tool and external API calls
Database operations
Infrastructure (servers, storage)

Knowing where money goes helps you prioritize optimization efforts.

Optimize Language Model UsageLink Copied

Language model calls are typically the most expensive part of running an agent. Here are ways to reduce these costs:

Use Smaller Models When Appropriate: Not every task needs your most powerful model. Simple questions might work fine with a smaller, cheaper model:

In[4]:

Code

def route_to_model(user_message):
    """Choose the right model for the task"""
    # Simple factual questions can use a smaller model
    if is_simple_question(user_message):
        return "claude-sonnet-4-5"  # Faster and cheaper
    else:
        return "claude-opus-4.1"  # More capable but more expensive

Cache Common Responses: If users frequently ask the same questions, cache the responses:

In[5]:

Code

import hashlib

response_cache = {}


def get_cached_response(user_message):
    """Check if we've answered this before"""
    message_hash = hashlib.md5(user_message.encode()).hexdigest()
    return response_cache.get(message_hash)


def cache_response(user_message, response):
    """Store response for future use"""
    message_hash = hashlib.md5(user_message.encode()).hexdigest()
    response_cache[message_hash] = response

This works well for factual questions that have stable answers. Don't cache responses for personalized or time-sensitive queries.

Reduce Token Usage: Language models charge per token (roughly per word). Shorter prompts and responses cost less:

Trim unnecessary context from prompts
Ask the model to be concise when appropriate
Remove redundant information from conversation history

Batch Operations: If your agent makes multiple API calls for one request, see if you can combine them:

In[6]:

Code

## Instead of three separate calls
response1 = model.generate("Question 1")
response2 = model.generate("Question 2")
response3 = model.generate("Question 3")

## Combine into one call
combined_prompt = """
Please answer these three questions:

1. Question 1
2. Question 2
3. Question 3
"""
response = model.generate(combined_prompt)

This reduces API overhead and often costs less than separate calls.

Set Budget AlertsLink Copied

Most API providers let you set spending limits or alerts. Configure these to notify you if costs exceed expected levels:

"Alert me if monthly API costs exceed $100"

This catches unexpected spikes before they become expensive surprises.

Review Costs RegularlyLink Copied

Make cost review part of your maintenance routine. Once a month, look at your spending:

Has it increased? Why?
Are there any surprises?
Are there new optimization opportunities?

Regular review helps you stay ahead of cost issues rather than reacting to them.