Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Checking and Refining the Agent's Reasoning › A More Complex Example

Michael BrenndoerferJune 19, 20253 min read

AI Agent HandbookAll chapters

Part 4 of 15Reasoning - Teaching the Agent to Think

Chapter 11Checking and Refining the Agent's Reasoning › A More Complex Example

Reading section 5 of 14 · A More Complex Example · 3 minSection 5 of 14 · 3 min

Section 5 of 14 · 3 min read

Checking and Refining the Agent's Reasoning › A More Complex ExampleOne section · Sections in this chapter5 / 14 ↓

Sections in this chapter14

Let's see verification in action with a trickier problem.

Problem: "You have a 3-gallon jug and a 5-gallon jug. How can you measure exactly 4 gallons of water?"

This is a classic puzzle that requires creative thinking. Let's see how an agent might solve it, then verify its solution.

Example (Claude Sonnet 4.5)Link Copied

In[4]:

Code

from anthropic import Anthropic

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

problem = """You have a 3-gallon jug and a 5-gallon jug. 
How can you measure exactly 4 gallons of water?"""

## Initial solution
initial_response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": f"{problem}\n\nThink through this step by step."}
    ],
)

solution = initial_response.content[0].text
print("Initial solution:", solution)

## Verification with detailed checking
verification = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": f"""You proposed this solution to the jug problem:
{solution}

Please verify by:

1. Walking through each step and tracking the water in each jug
2. Confirming you end with exactly 4 gallons
3. Checking if there's a simpler solution

Show your verification work clearly.""",
        }
    ],
)

print("\nVerification:", verification.content[0].text)

Out[4]:

Console

Initial solution: I need to measure exactly 4 gallons using a 3-gallon jug and a 5-gallon jug.

Let me work through this step-by-step:

**Solution:**

1. **Fill the 5-gallon jug completely**
   - 3-gal jug: 0 gallons
   - 5-gal jug: 5 gallons

2. **Pour from the 5-gallon jug into the 3-gallon jug**
   - 3-gal jug: 3 gallons (full)
   - 5-gal jug: 2 gallons (remaining)

3. **Empty the 3-gallon jug**
   - 3-gal jug: 0 gallons
   - 5-gal jug: 2 gallons

4. **Pour the 2 gallons from the 5-gallon jug into the 3-gallon jug**
   - 3-gal jug: 2 gallons
   - 5-gal jug: 0 gallons

5. **Fill the 5-gallon jug completely again**
   - 3-gal jug: 2 gallons
   - 5-gal jug: 5 gallons

6. **Pour from the 5-gallon jug into the 3-gallon jug (which already has 2 gallons)**
   - The 3-gallon jug can only take 1 more gallon
   - 3-gal jug: 3 gallons (full)
   - 5-gal jug: **4 gallons** ✓

**Result:** The 5-gallon jug now contains exactly 4 gallons!


Verification: # Verification of the Jug Problem Solution

Let me carefully track each step:

## Step-by-Step Verification

| Step | Action | 3-gal jug | 5-gal jug | ✓/✗ |
|------|--------|-----------|-----------|-----|
| Start | Initial state | 0 | 0 | ✓ |
| 1 | Fill 5-gal jug | 0 | 5 | ✓ |
| 2 | Pour 5-gal → 3-gal (pour 3) | 3 | 2 | ✓ |
| 3 | Empty 3-gal jug | 0 | 2 | ✓ |
| 4 | Pour 5-gal → 3-gal (pour 2) | 2 | 0 | ✓ |
| 5 | Fill 5-gal jug | 2 | 5 | ✓ |
| 6 | Pour 5-gal → 3-gal (pour 1) | 3 | 4 | ✓ |

## Verification Checks

**1. Each step is valid:**
- ✓ Step 2: 3-gal jug empty, can accept 3 gallons from 5-gal
- ✓ Step 4: Pouring 2 gallons into empty 3-gal jug works
- ✓ Step 6: 3-gal jug has 2 gallons, can only accept 1 more (5 - 1 = 4)

**2. Final result:**
- ✓ **5-gallon jug contains exactly 4 gallons**

**3. Simpler solution?**

Let me check an alternative:

| Step | Action | 3-gal jug | 5-gal jug |
|------|--------|-----------|-----------|
| 1 | Fill 3-gal | 3 | 0 |
| 2 | Pour 3-gal → 5-gal | 0 | 3 |
| 3 | Fill 3-gal again | 3 | 3 |
| 4 | Pour 3-gal → 5-gal (fills it) | 1 | 5 |
| 5 | Empty 5-gal | 1 | 0 |
| 6 | Pour 3-gal → 5-gal | 0 | 1 |
| 7 | Fill 3-gal | 3 | 1 |
| 8 | Pour 3-gal → 5-gal | 0 | 4 |

This takes **8 steps** vs the original **6 steps**.

## Conclusion

✅ **The original solution is VERIFIED and CORRECT**
- It successfully produces exactly 4 gallons in the 5-gallon jug
- It uses only 6 steps
- It appears to be the optimal (shortest) solution

The agent might initially propose:

Here's one way to measure 4 gallons:

1. Fill the 5-gallon jug completely
2. Pour from the 5-gallon jug into the 3-gallon jug (leaving 2 gallons in the 5-gallon jug)
3. Empty the 3-gallon jug
4. Pour the 2 gallons from the 5-gallon jug into the 3-gallon jug
5. Fill the 5-gallon jug again
6. Pour from the 5-gallon jug into the 3-gallon jug until it's full (this adds 1 gallon)
7. The 5-gallon jug now contains exactly 4 gallons

During verification, the agent walks through each step:

Let me verify step by step:

Start: 5-gal = 0, 3-gal = 0

Step 1: Fill 5-gallon jug
$\to$ 5-gal = 5, 3-gal = 0 ✓

Step 2: Pour into 3-gallon jug
$\to$ 5-gal = 2, 3-gal = 3 ✓

Step 3: Empty 3-gallon jug
$\to$ 5-gal = 2, 3-gal = 0 ✓

Step 4: Pour 2 gallons into 3-gallon jug
$\to$ 5-gal = 0, 3-gal = 2 ✓

Step 5: Fill 5-gallon jug
$\to$ 5-gal = 5, 3-gal = 2 ✓

Step 6: Pour into 3-gallon jug (which has 2 gallons, so can take 1 more)
$\to$ 5-gal = 4, 3-gal = 3 ✓

Result: 4 gallons in the 5-gallon jug. Solution verified!

This detailed verification confirms the solution works. The agent caught no errors here, but the systematic check builds confidence.