Maintenance and Updates › Making Safe Updates

Michael BrenndoerferAugust 22, 20253 min read
AI Agent HandbookAll chapters
Part 14 of 15Deployment and Operations - From Prototype to Production
Chapter 41Maintenance and Updates › Making Safe Updates
Reading section 5 of 19 · Making Safe Updates · 3 minSection 5 of 19 · 3 min

Book contents

AI Agent Handbook15 parts · 44 chaptersView the whole book →
Active chapter 41 of 44

Section 5 of 19 · 3 min read

Maintenance and Updates › Making Safe UpdatesOne section · Sections in this chapter5 / 19 ↓
Sections in this chapter19

Let's walk through the process of updating your agent safely. We'll use a concrete example: updating to a new version of the language model.

Step 1: Identify What's Changing

Suppose Anthropic releases Claude Sonnet 4.6 with improved reasoning capabilities. You want to upgrade from Claude Sonnet 4.5. First, read the release notes. What changed? Are there new features? Breaking changes? Different behavior?

Understanding what changed helps you know what to test.

Step 2: Update in Development

Make the change in your development environment first. This might be as simple as changing one line:

In[2]:
Code
## Before
response = client.messages.create(
    model="claude-sonnet-4-5", max_tokens=1024, messages=messages
)

## After
response = client.messages.create(
    model="claude-sonnet-4.6", max_tokens=1024, messages=messages
)

Step 3: Test Thoroughly

Now test the updated agent with a variety of inputs. Use your test cases from Chapter 11 (Evaluation). Try edge cases. See if the new model handles your prompts the same way.

Pay special attention to:

  • Core functionality: Does the agent still accomplish its primary tasks?
  • Tool usage: Does it call tools appropriately?
  • Response quality: Are responses still helpful and accurate?
  • Edge cases: Does it handle unusual inputs gracefully?
  • Performance: Is it faster or slower than before?

If you find issues, you have options. Maybe you need to adjust your prompts for the new model. Maybe the new model isn't ready yet and you should wait. Maybe the issues are minor and acceptable.

Step 4: Deploy to Staging

Once development testing looks good, deploy to staging. This tests the update in an environment that closely matches production.

Run the same tests again. Also test the deployment process itself. Did the update deploy cleanly? Are environment variables set correctly? Does the agent start up properly?

Step 5: Monitor and Deploy to Production

If staging looks good, deploy to production. But don't walk away. Watch your monitoring dashboard closely for the first few hours. Check:

  • Error rate: Has it increased?
  • Response time: Has it changed?
  • User feedback: Are users reporting problems?
  • Logs: Do you see unexpected errors or warnings?

If something looks wrong, you can roll back. Most deployment platforms make it easy to revert to the previous version quickly.

Step 6: Document the Change

After the update is stable, document what you changed and why. This helps future maintenance:

2025-11-10: Updated to Claude Sonnet 4.6
- Reason: Improved reasoning capabilities, 15% faster responses
- Changes: Updated model parameter in assistant.py
- Testing: All test cases passed, staging ran for 24 hours with no issues
- Monitoring: Error rate unchanged, average response time decreased from 2.1s to 1.8s

This changelog becomes a valuable reference when troubleshooting future issues or planning further updates.