Testing the Agent with Examples › A Complete Testing Example

Michael BrenndoerferAugust 2, 20251 min read
AI Agent HandbookAll chapters
Part 11 of 15Evaluation - Measuring Your Agent's Performance
Chapter 31Testing the Agent with Examples › A Complete Testing Example
Reading section 8 of 11 · A Complete Testing Example · 1 minSection 8 of 11 · 1 min

Book contents

Section 8 of 11 · 1 min read

Testing the Agent with Examples › A Complete Testing ExampleOne section · Sections in this chapter8 / 11 ↓

Let's put everything together. Here's a complete, runnable example of testing our personal assistant:

In[2]:
Code
## Using Claude Sonnet 4.5 for comprehensive agent testing

## Initialize your agent (implementation from previous chapters)
agent = PersonalAssistant(
    model="claude-sonnet-4-5", tools=[calculator, weather_api, memory_manager]
)

## Define test suite
test_suite = [
    {
        "id": "basic_question",
        "input": "What is the capital of France?",
        "expected_output": "Paris",
        "category": "knowledge",
    },
    {
        "id": "tool_calculator",
        "input": "Calculate 789 times 456",
        "expected_behavior": "uses_calculator",
        "expected_output": "359,784",
        "category": "tool_use",
    },
    {
        "id": "memory_store",
        "input": "Remember that I prefer coffee in the morning",
        "expected_behavior": "stores_to_memory",
        "category": "memory",
    },
    {
        "id": "memory_retrieve",
        "input": "What do I like to drink in the morning?",
        "expected_output": "coffee",
        "category": "memory",
        "requires": ["memory_store"],
    },
]

## Run tests
results = run_test_suite(agent, test_suite)

## Generate summary
summarize_results(results)

## Save for historical tracking
save_test_run(results, version_name="v1.2")

When you run this, you'll get immediate feedback on how your agent performs, a summary of results by category, and a saved record you can compare against future test runs.