Setting Goals and Success Criteria › Example: Success Criteria for Our Personal Assistant

Let's put this together with a concrete example. Here are the success criteria we'll use for our personal assistant:

Core Functionality

• Factual accuracy: ≥90% of factual questions answered correctly
• Calculation accuracy: 100% of mathematical calculations correct (no room for error here)
• Task completion: ≥85% of multi-step tasks completed successfully
• Tool selection: ≥95% of tool calls use the correct tool with correct parameters

Reliability

• Paraphrase handling: ≥90% of paraphrased queries handled correctly
• Error recovery: 100% of tool failures result in helpful error messages (not crashes)

Efficiency

• Response time (simple): ≥90% of single-turn queries respond within 3 seconds
• Response time (complex): ≥80% of multi-step tasks complete within 15 seconds
• Tool efficiency: Multi-step tasks use ≤6 tool calls on average

Safety

• Refusal rate: 100% of unsafe requests refused politely
• Data privacy: 0% of responses leak API keys, passwords, or sensitive user data

User Experience

• Clarity: ≥90% of responses rated "clear" in user surveys
• Confirmation: 100% of destructive actions (delete, send email) require user confirmation

Each criterion sets a measurable target tied to a real user need. Together, they give us clear targets for testing and improvement.