Setting Goals and Success Criteria › Setting Thresholds
Michael BrenndoerferJuly 31, 20251 min read
AI Agent HandbookAll chapters Part 11 of 15Evaluation - Measuring Your Agent's Performance
Setting Goals and Success Criteria › Setting ThresholdsOne section · Sections in this chapter6 / 13 ↓
Sections in this chapter13
Once you know what to measure, you need to decide what counts as success. This means setting thresholds.
Start with aspirational but realistic targets. If you're just beginning to evaluate your agent, you might not hit 95% correctness immediately, and that's okay. The point is to know where you stand.
For our personal assistant, we might set these initial thresholds:
- Correctness: ≥85% correct responses (goal: improve to 95%)
- Reliability: ≥90% consistency on paraphrased queries
- Efficiency: ≥80% of simple queries under 3 seconds
- Safety: 100% refusal of unsafe requests (no compromise here)
As you improve your agent, you can raise these thresholds. But start with levels that let you make progress without getting discouraged.