Testing the Agent with Examples › Glossary

Test Case: A specific input, expected behavior, and expected output used to verify that an agent works correctly. Think of it like a practice problem that tests whether your agent learned a particular skill.

Test Suite: A collection of related test cases that together verify multiple aspects of an agent's functionality. The suite should cover all the major capabilities your agent needs.

Pass Rate: The percentage of tests that succeed in a test run. A pass rate of 80% means 8 out of 10 tests passed, giving you a quantitative measure of your agent's performance.

Test Dependency: When one test requires another test to run successfully first. For example, testing memory retrieval requires that memory storage worked first.

Test Fixture: Setup code that prepares the environment for testing, such as initializing an agent or loading test data. Fixtures help you avoid repeating setup code in every test.

Regression: When a change to your agent causes previously passing tests to fail. Tracking regressions helps ensure that improvements don't accidentally break existing functionality.