Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Using a Language Model in Code › Controlling the Output

Michael BrenndoerferJune 27, 20252 min read

AI Agent HandbookAll chapters

Part 2 of 15Language Models - The Brain of the Agent

Chapter 5Using a Language Model in Code › Controlling the Output

Reading section 8 of 15 · Controlling the Output · 2 minSection 8 of 15 · 2 min

Section 8 of 15 · 2 min read

Using a Language Model in Code › Controlling the OutputOne section · Sections in this chapter8 / 15 ↓

Sections in this chapter15

Language models are probabilistic. They don't always give the same answer to the same question. Traditionally, you could control this behavior with parameters like temperature, but this has changed with newer models.

Important Note about GPT-5: OpenAI has removed the temperature parameter from GPT-5 to standardize response behavior and improve reliability. If you need temperature control, use Claude Sonnet 4.5 or Gemini 2.5 instead.

Example (Claude Sonnet 4.5 with temperature control)

In[2]:

Code

from anthropic import Anthropic

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

## More creative/random responses
## Using Claude Sonnet 4.5 for its temperature control capabilities
response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    temperature=1.0,  # Higher = more random (range: 0 to 1 for Claude)
    messages=[
        {
            "role": "user",
            "content": "Write a creative opening line for a story.",
        }
    ],
)
print(response.content[0].text)

## More deterministic/focused responses
response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    temperature=0.0,  # Lower = more deterministic
    messages=[{"role": "user", "content": "What is 2 + 2?"}],
)

print(response.content[0].text)

Temperature (when available) controls randomness. At 0.0, the model always picks the most likely next word, making responses consistent and predictable. At higher values (up to 1.0 for Claude, 2.0 for Gemini), it samples from a wider range of possibilities, making responses more creative but less reliable.

For factual questions and tasks requiring consistency, use low temperature (0.0 to 0.3). For creative tasks like writing or brainstorming, use higher temperature (0.7 to 1.0).

Claude Sonnet 4.5 and Gemini 2.5 both support temperature parameters, making them excellent choices when you need this level of control in your applications.

Other useful parameters (availability varies by model):

max_tokens: Limits response length (useful for controlling costs and keeping responses concise). Available in all current models.
top_p: Alternative to temperature for controlling randomness (typically use one or the other, not both). Available in Claude and Gemini.
system: Sets the model's behavior and personality (Claude uses a separate parameter; GPT-5 and Gemini use message roles).

For most use cases with GPT-5, you'll primarily adjust max_tokens and system messages. For Claude Sonnet 4.5 and Gemini 2.5, you have additional control through temperature and top_p parameters.