Step-by-Step Problem Solving (Chain-of-Thought) › When Chain-of-Thought Helps Most
AI Agent HandbookAll chapters Step-by-Step Problem Solving (Chain-of-Thought) › When Chain-of-Thought Helps MostOne section · Sections in this chapter5 / 13 ↓
Chain-of-thought reasoning isn't necessary for every task. If you're asking for a fact ("What's the capital of France?") or a simple transformation ("Translate this to Spanish"), you don't need explicit reasoning steps. The model can answer directly.
But chain-of-thought becomes valuable when:
The problem has multiple steps: Each step builds on the previous one, and skipping a step leads to errors.
The answer isn't obvious: The problem requires analysis, not just recall.
You need to verify the logic: Being able to check the reasoning is as important as getting the right answer.
The model tends to make mistakes: If you notice the model getting certain types of questions wrong, adding chain-of-thought often helps.
Example: A Tricky Riddle
Riddles are a great test case because they require careful thinking. Let's try one:
Example (GPT-5)
from openai import OpenAI
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
riddle = """I speak without a mouth and hear without ears.
I have no body, but I come alive with wind. What am I?
Let's think through this step by step."""
## Using GPT-5 for reasoning through riddles and logic puzzles
response = client.chat.completions.create(
model="gpt-5", messages=[{"role": "user", "content": riddle}]
)
print(response.choices[0].message.content)An echo. It “speaks” by repeating sounds without a mouth, “hears” by receiving them without ears, has no body, and it exists only as sound traveling through air—so it metaphorically “comes alive” with the wind.
The model might respond:
Let's think through this step by step.
Step 1: Analyze the first clue - "I speak without a mouth"
This suggests something that makes sound but isn't alive or doesn't have a physical mouth.
Step 2: Consider the second clue - "hear without ears"
This means it can respond to or carry sound without having ears. This reinforces that it's not a living thing.
Step 3: Examine the third clue - "I have no body"
This confirms it's not a physical creature or person.
Step 4: Look at the final clue - "I come alive with wind"
This is the key clue. Something that needs wind to function and creates sound.
Step 5: Put it together
Something without a body that makes sound and requires wind... this describes an echo. Sound travels through air (wind), bounces off surfaces, and returns as an echo. An echo "speaks" (produces sound) without a mouth, "hears" (responds to sound) without ears, has no physical body, and exists because of sound waves traveling through air.
Answer: An echoBy working through each clue systematically, the model arrives at the correct answer with clear reasoning you can follow.