Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Checking and Refining the Agent's Reasoning › The Power of Self-Verification

Michael BrenndoerferJune 19, 20252 min read

AI Agent HandbookAll chapters

Part 4 of 15Reasoning - Teaching the Agent to Think

Chapter 11Checking and Refining the Agent's Reasoning › The Power of Self-Verification

Reading section 3 of 14 · The Power of Self-Verification · 2 minSection 3 of 14 · 2 min

Section 3 of 14 · 2 min read

Checking and Refining the Agent's Reasoning › The Power of Self-VerificationOne section · Sections in this chapter3 / 14 ↓

Sections in this chapter14

Let's start with a simple example. Here's an agent solving a word problem:

Problem: "A store has 15 apples. They sell 8 apples in the morning and 6 apples in the afternoon. How many apples are left?"

First attempt:

Let me solve this step by step:

1. Start with 15 apples
2. Sell 8 in the morning: 15 - 8 = 7 apples left
3. Sell 6 in the afternoon: 7 - 6 = 1 apple left

Answer: 1 apple remains.

This looks reasonable. But what if we prompt the agent to verify its answer?

Example (GPT-5)Link Copied

In[3]:

Code

from openai import OpenAI

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

## Initial problem solving
problem = """A store has 15 apples. They sell 8 apples in the morning 
and 6 apples in the afternoon. How many apples are left?"""

initial_response = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": f"{problem}\n\nSolve this step by step."}],
)

initial_answer = initial_response.choices[0].message.content
print("Initial answer:", initial_answer)

## Now ask the agent to verify
verification_prompt = f"""You solved this problem:
{problem}

Your answer was:
{initial_answer}

Please verify your answer by:

1. Checking each calculation
2. Confirming the logic makes sense
3. Considering if you missed anything

Are you confident in this answer? If you find any issues, provide a corrected solution."""

verification_response = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": verification_prompt}],
)

print("\nVerification:", verification_response.choices[0].message.content)

Out[3]:

Console

Initial answer: - Start with 15 apples.
- After selling 8 in the morning: 15 − 8 = 7 apples left.
- After selling 6 in the afternoon: 7 − 6 = 1 apple left.

Answer: 1 apple.


Verification: 1) Check each calculation:
- Morning: 15 − 8 = 7. Correct.
- Afternoon: 7 − 6 = 1. Correct.
- Combined check: 15 − (8 + 6) = 15 − 14 = 1. Matches.

2) Logic check:
- They had enough apples for each sale (7 left after morning, so selling 6 in the afternoon is feasible).
- Sequential subtraction or combined subtraction both yield the same result.

3) Consider what might be missing:
- No mention of restocking, spoilage, or returns, so subtracting the total sold from the initial stock is appropriate.

Conclusion: The answer is correct. 1 apple left. I’m confident in this result.

The verification step might reveal:

Let me verify my calculations:

1. Start: 15 apples ✓
2. After morning sales: 15 - 8 = 7 ✓
3. After afternoon sales: 7 - 6 = 1 ✓

Wait, let me also check if the problem is even possible:
- Total sold: 8 + 6 = 14 apples
- Started with: 15 apples
- 15 - 14 = 1 apple

Yes, the answer checks out. We have 1 apple remaining.

In this case, the answer was correct. But the verification process adds confidence. The agent didn't just solve the problem; it confirmed the solution makes sense.