Setting Goals and Success Criteria › Making Criteria Measurable

Good success criteria are specific and measurable. "The agent should be good" isn't useful. "The agent should correctly answer 90% of factual questions from our test set" is.

Turn each vague goal into a measurable criterion:

Vague: "The agent should usually work."

Measurable: "The agent should successfully complete tasks in at least 85% of test cases."

Vague: "Responses should be fast."

Measurable: "90% of single-turn queries should receive responses within 3 seconds."

Vague: "The agent should handle errors well."

Measurable: "When a tool call fails, the agent should provide a helpful error message to the user and suggest alternatives 100% of the time."

Notice the pattern: measurable criteria include numbers and clear conditions. You should be able to look at an interaction and definitively say whether it meets the criterion.