Setting Goals and Success Criteria › Start with User Goals

Michael BrenndoerferJuly 31, 20251 min read
AI Agent HandbookAll chapters
Part 11 of 15Evaluation - Measuring Your Agent's Performance
Chapter 30Setting Goals and Success Criteria › Start with User Goals
Reading section 3 of 13 · Start with User Goals · 1 minSection 3 of 13 · 1 min

Book contents

AI Agent Handbook15 parts · 44 chaptersView the whole book →
Active chapter 30 of 44

Section 3 of 13 · 1 min read

Setting Goals and Success Criteria › Start with User GoalsOne section · Sections in this chapter3 / 13 ↓
Sections in this chapter13

The best success criteria emerge from understanding what users need from your agent. Our personal assistant exists to help users accomplish tasks, so we should define success in terms of those tasks.

Start by listing the main capabilities you've given your agent. For our assistant, that might include:

  • Answering factual questions
  • Performing calculations
  • Managing calendar events
  • Retrieving information from memory
  • Planning and executing multi-step tasks
  • Searching the web for current information

For each capability, ask: What would a successful interaction look like from the user's perspective?

Example: Calendar Management

When a user says, "Add a dentist appointment next Friday at 3 PM," success means:

  • The event is created in the calendar
  • The date is correct (next Friday, not this Friday or some other day)
  • The time is correct (3 PM, not 3 AM or 2 PM)
  • The description is accurate ("dentist appointment")
  • The user receives confirmation

Notice how specific this is. We're not just saying "the agent should handle calendar requests." We're defining exactly what "handle" means in measurable terms.