Checking and Refining the Agent's Reasoning › When Verification Catches Mistakes
AI Agent HandbookAll chapters Checking and Refining the Agent's Reasoning › When Verification Catches MistakesOne section · Sections in this chapter6 / 14 ↓
Verification really shines when the initial reasoning has flaws. Consider this scenario:
Problem: "A train travels 60 miles in 1 hour. At this rate, how far will it travel in 90 minutes?"
Initial answer: "60 miles in 60 minutes, so 90 miles in 90 minutes."
This is wrong. The agent incorrectly assumed a 1:1 ratio between minutes and miles. Let's see how verification helps:
## Using the verification pattern from earlier
verification_prompt = """You said the train travels 90 miles in 90 minutes.
Let's verify this:
1. What is the train's speed in miles per minute?
2. How many miles would it travel in 90 minutes at that speed?
3. Does your answer match your calculation?
Check your work carefully."""The agent might respond:
Let me recalculate:
1. Speed: 60 miles / 60 minutes = 1 mile per minute
2. Distance in 90 minutes: 1 mile/minute $\times$ 90 minutes = 90 miles
Wait, that gives me 90 miles, which matches my answer. Let me verify once more:
- 60 miles in 60 minutes = 1 mile per minute ✓
- 90 minutes $\times$ 1 mile per minute = 90 miles ✓
Actually, my answer was correct. The train travels 90 miles in 90 minutes.In this case, the agent's initial answer was actually right (my example was misleading). But the verification process forced it to show its work clearly, making the reasoning transparent.
Let's try a problem where the initial answer is genuinely wrong:
Problem: "If 5 machines can produce 5 widgets in 5 minutes, how many machines are needed to produce 100 widgets in 100 minutes?"
Initial answer: "20 machines (scaling up proportionally)."
This is a classic trick question. Let's verify:
verification_prompt = """You said 20 machines are needed.
Before confirming, let's think about this carefully:
1. How many widgets does ONE machine produce in 5 minutes?
2. How many widgets does ONE machine produce in 100 minutes?
3. How many machines do you need to produce 100 widgets in 100 minutes?
Work through this step by step."""The verification reveals:
Let me reconsider:
1. If 5 machines produce 5 widgets in 5 minutes, then each machine produces 1 widget in 5 minutes.
2. In 100 minutes, one machine produces: 100 $\div$ 5 = 20 widgets
3. To produce 100 widgets in 100 minutes: 100 widgets $\div$ 20 widgets per machine = 5 machines
Wait, I made an error. The answer is 5 machines, not 20.
Each machine produces at a rate of 1 widget per 5 minutes. In 100 minutes, each machine makes 20 widgets. So 5 machines make 100 widgets in 100 minutes.
Corrected answer: 5 machines.The verification caught the mistake. By breaking down the problem into smaller questions, the agent reconsidered its initial (incorrect) assumption.