Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Step-by-Step Problem Solving (Chain-of-Thought) › Teaching Through Examples: Few-Shot Chain-of-Thought

Michael BrenndoerferJune 17, 20252 min read

AI Agent HandbookAll chapters

Part 4 of 15Reasoning - Teaching the Agent to Think

Chapter 10Step-by-Step Problem Solving (Chain-of-Thought) › Teaching Through Examples: Few-Shot Chain-of-Thought

Reading section 6 of 13 · Teaching Through Examples: Few-Shot Chain-of-Thought · 2 minSection 6 of 13 · 2 min

Section 6 of 13 · 2 min read

Step-by-Step Problem Solving (Chain-of-Thought) › Teaching Through Examples: Few-Shot Chain-of-ThoughtOne section · Sections in this chapter6 / 13 ↓

Sections in this chapter13

Remember few-shot prompting from the previous chapter? You can combine that technique with chain-of-thought by showing the model examples of step-by-step reasoning. This is especially powerful for tasks where you want consistent reasoning patterns.

Instead of showing input-output pairs, you show input-reasoning-output triplets. The model learns what to answer and how to think about the problem.

Example (GPT-5)Link Copied

Let's say you're building a feature where your assistant helps evaluate whether claims are supported by evidence:

In[5]:

Code

from openai import OpenAI

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

prompt = """Determine if the claim is supported by the evidence. Show your reasoning.

Example 1:
Claim: "Exercise improves mental health"
Evidence: "A study of 1,000 participants found that those who exercised 30 minutes daily reported 25% lower anxiety levels than those who didn't exercise."
Reasoning:
- The evidence comes from a study with a large sample size (1,000 participants)
- It shows a specific, measurable benefit (25% lower anxiety)
- Anxiety is a component of mental health
- The evidence directly relates to the claim
Conclusion: Supported

Example 2:
Claim: "Coffee causes heart disease"
Evidence: "Some people who drink coffee have reported heart palpitations."
Reasoning:
- The evidence is anecdotal ("some people reported")
- Heart palpitations are not the same as heart disease
- No causal relationship is established (correlation vs causation)
- The evidence is too weak to support the strong claim
Conclusion: Not supported

Now evaluate this:
Claim: "Reading before bed improves sleep quality"
Evidence: "A survey found that 60% of people who read before bed felt they slept better."
"""

## Using GPT-5 for analytical reasoning with few-shot examples
response = client.chat.completions.create(
    model="gpt-5", messages=[{"role": "user", "content": prompt}]
)

print(response.choices[0].message.content)

Out[5]:

Console

Reasoning:
- The evidence is a self-reported survey; sample size and methodology are unspecified, raising concerns about reliability and bias.
- It lacks a control or comparison group (e.g., people who don’t read before bed), so we can’t infer that reading caused the improvement.
- “Felt they slept better” is subjective and may not reflect objective sleep quality; it also leaves 40% who did not report improvement.
- The claim implies a general causal effect, while the evidence only shows that a subset of readers perceive a benefit, without establishing causation.

Conclusion: Not supported. The evidence is insufficient to substantiate the causal claim.

The model will follow the reasoning pattern you demonstrated:

Reasoning:
- The evidence comes from a survey, which captures self-reported data
- 60% is a majority, suggesting a notable correlation
- "Felt they slept better" is subjective, not an objective measure of sleep quality
- The evidence shows correlation but doesn't prove causation (other factors could be involved)
- The sample size and methodology aren't specified, which limits confidence
Conclusion: Partially supported (shows correlation but not causation)

By providing examples of good reasoning, you've taught the model how to approach this type of analysis.