Testing the Agent with Examples › Making Tests Maintainable

As your agent evolves, your test suite needs to keep pace. Here are some practices that keep tests useful over time:

Use clear test IDs and descriptions. Six months from now, test_001 won't mean anything to you. But memory_birthday_storage tells you exactly what's being tested.

Group related tests. Keep all memory tests together, all tool tests together. This makes it easier to find and update tests when you modify a specific capability.

Document edge cases. When you discover a bug or unexpected behavior, write a test for it. Your test suite becomes a record of problems you've solved.

Keep expected outputs realistic. Don't require exact text matches. Language models vary their responses, and that's okay. Test for the presence of key information, not specific phrasings.

Review failed tests carefully. Sometimes a failed test means your agent is broken. But sometimes it means your test expectations were wrong. Be willing to update tests as your understanding improves.