Claude Cookbook

Automatic context compaction

Manage context limits in long-running agentic workflows by automatically compressing conversation history.

Pedram Navid
Pedram Navid
@PedramNavid
Published on November 24, 2025
Was this page helpful?

Automatic Context Compaction

Long-running agentic tasks can often exceed context limits. Tool heavy workflows or long conversations quickly consume the token context window. In Effective Context Engineering for AI Agents, we discussed how managing context can help avoid performance degradation and context rot.

The Anthropic Python SDK's tool runner can help manage this context: when your loop decides the conversation has grown too large, one call to runner.compact_before_next_turn() has the API summarize the history server-side and the runner carries on from that summary, allowing tasks to continue beyond the model's context window limit.

In this cookbook, we'll demonstrate context compaction through an agentic customer service workflow. Imagine you've built an AI customer service agent tasked with processing a queue of support tickets. For each ticket, you must classify the issue, search the knowledge base, set priority, route to the appropriate team, draft a response, and mark it complete. As you process ticket after ticket, the conversation history fills with classifications, knowledge base searches, and drafted responses—quickly consuming thousands of tokens.

What is Context Compaction?

When building agentic workflows with tool use, conversations can grow very large as the agent iterates on complex tasks. Runner-driven compaction manages this in four steps:

  1. Your loop monitors token usage per turn in the conversation
  2. When your threshold is exceeded, you call runner.compact_before_next_turn()
  3. Once the current turn (including its tool calls) finishes, the runner sends a compaction request. The API summarizes the conversation server-side and returns a single compaction block (stop_reason == "compaction")
  4. The runner replaces the conversation history with that block and continues the task with the compressed context

By the end of this cookbook, you'll be able to:

  • Understand how to effectively manage context limits in iterative workflows
  • Write agents that leverage automatic context compaction
  • Design workflows that maintain focus across multiple iterations

Prerequisites

Before following this guide, ensure you have:

Required Knowledge

  • Basic understanding of agentic patterns and tool calling

Required Tools

  • Python 3.11 or higher
  • Anthropic API key
  • Anthropic SDK >= 2.0.0 (uses the compact-2026-09-04 beta)

Just want compaction to happen automatically? Server-side compaction via context_management compacts on its own once input tokens cross a trigger you set (50k tokens or more), with no loop logic at all.

This cookbook covers runner-driven compaction, where your code decides when to compact: useful for low thresholds, natural item boundaries (compact after every N tickets), or custom summarization instructions.

Setup

First, install the required dependencies:

# %pip install -qU anthropic python-dotenv

Note: Ensure your .env file contains:

ANTHROPIC_API_KEY=your_key_here

Load your environment variables and configure the client. We also load a helper utility to visualize Claude message responses.

from dotenv import load_dotenv
load_dotenv()
MODEL = "claude-sonnet-5-5"

Setting the Stage

In utils/customer_service_tools.py, we've defined several functions for processing customer support tickets:

  • get_next_ticket() - Retrieves the next unprocessed ticket from the queue
  • classify_ticket(ticket_id, category) - Categorizes issues as billing, technical, account, product, or shipping
  • search_knowledge_base(query) - Finds relevant help articles and solutions
  • set_priority(ticket_id, priority) - Assigns priority levels (low, medium, high, urgent)
  • route_to_team(ticket_id, team) - Routes tickets to the appropriate support team
  • draft_response(ticket_id, response_text) - Creates customer-facing responses
  • mark_complete(ticket_id) - Finalizes processed tickets

For a customer service agent, these tools enable processing tickets systematically. Each ticket requires classification, research, prioritization, routing, and response drafting. When processing 20-30 tickets in sequence, the conversation history fills with tool results from every classification, every knowledge base search, and every drafted response, causing linear token growth.

The beta_tool decorator is used on the tools to make them accessible to the Claude agent. The decorator extracts the function arguments and docstring and provides these to Claude as tool metadata.

import anthropic
from anthropic import beta_tool
@beta_tool
def get_next_ticket() -> dict:
"""Retrieve the next unprocessed support ticket from the queue."""
...
import anthropic
from utils.customer_service_tools import (
classify_ticket,
draft_response,
get_next_ticket,
initialize_ticket_queue,
mark_complete,
route_to_team,
search_knowledge_base,
set_priority,
)
client = anthropic.Anthropic()
tools = [
get_next_ticket,
classify_ticket,
search_knowledge_base,
set_priority,
route_to_team,
draft_response,
mark_complete,
]

Baseline: Running Without Compaction

Let's start with a realistic customer service scenario: Processing a queue of support tickets.

The workflow looks like this:

For Each Ticket:

  1. Fetch the ticket using get_next_ticket()
  2. Classify the issue category (billing, technical, account, product, shipping)
  3. Search the knowledge base for relevant information
  4. Set appropriate priority (low, medium, high, urgent)
  5. Route to the correct team
  6. Draft a customer response
  7. Mark the ticket complete
  8. Move to the next ticket

The Challenge: With 5 tickets in the queue, and each requiring 7 tool calls, Claude will make 35 or more tool calls. The results from each step including classification knowledge base search, and drafted responses accumulate in the conversation history. Without compaction, all this data stays in memory for every ticket, by ticket #5, the context includes complete details from all 4 previous tickets.

Let's run this workflow without compaction first and observe what happens:

from anthropic.types.beta import BetaMessageParam
num_tickets = 5
initialize_ticket_queue(num_tickets)
messages: list[BetaMessageParam] = [
{
"role": "user",
"content": f"""You are an AI customer service agent. Your task is to process support tickets from a queue.
For EACH ticket, you must complete ALL these steps:
1. **Fetch ticket**: Call get_next_ticket() to retrieve the next unprocessed ticket
2. **Classify**: Call classify_ticket() to categorize the issue (billing/technical/account/product/shipping)
3. **Research**: Call search_knowledge_base() to find relevant information for this ticket type
4. **Prioritize**: Call set_priority() to assign priority (low/medium/high/urgent) based on severity
5. **Route**: Call route_to_team() to assign to the appropriate team
6. **Draft**: Call draft_response() to create a helpful customer response using KB information
7. **Complete**: Call mark_complete() to finalize this ticket
8. **Continue**: Immediately fetch the next ticket and repeat
IMPORTANT RULES:
- Process tickets ONE AT A TIME in sequence
- Complete ALL 7 steps for each ticket before moving to the next
- Keep fetching and processing tickets until you get an error that the queue is empty
- There are {num_tickets} tickets total - process all of them
- Be thorough but efficient
Begin by fetching the first ticket.""",
}
]
total_input = 0
total_output = 0
turn_count = 0
runner = client.beta.messages.tool_runner(
model=MODEL,
max_tokens=4096,
tools=tools,
messages=messages,
)
for message in runner:
messages_list = list(runner._params["messages"])
turn_count += 1
total_input += message.usage.input_tokens
total_output += message.usage.output_tokens
print(
f"Turn {turn_count:2d}: Input={message.usage.input_tokens:7,} tokens | "
f"Output={message.usage.output_tokens:5,} tokens | "
f"Messages={len(messages_list):2d} | "
f"Cumulative In={total_input:8,}"
)
print(f"\n{'=' * 60}")
print("BASELINE RESULTS (NO COMPACTION)")
print(f"{'=' * 60}")
print(f"Total turns: {turn_count}")
print(f"Input tokens: {total_input:,}")
print(f"Output tokens: {total_output:,}")
print(f"Total tokens: {total_input + total_output:,}")
print(f"{'=' * 60}")
Turn  1: Input=  1,732 tokens | Output=   30 tokens | Messages= 1 | Cumulative In=   1,732

Turn  2: Input=  1,964 tokens | Output=  318 tokens | Messages= 3 | Cumulative In=   3,696

Turn  3: Input=  2,881 tokens | Output=  378 tokens | Messages= 5 | Cumulative In=   6,577

Turn  4: Input=  3,516 tokens | Output=   30 tokens | Messages= 7 | Cumulative In=  10,093

Turn  5: Input=  3,744 tokens | Output=  313 tokens | Messages= 9 | Cumulative In=  13,837

Turn  6: Input=  4,875 tokens | Output=  409 tokens | Messages=11 | Cumulative In=  18,712

Turn  7: Input=  5,538 tokens | Output=   30 tokens | Messages=13 | Cumulative In=  24,250

Turn  8: Input=  5,758 tokens | Output=  309 tokens | Messages=15 | Cumulative In=  30,008

Turn  9: Input=  6,619 tokens | Output=  443 tokens | Messages=17 | Cumulative In=  36,627

Turn 10: Input=  7,316 tokens | Output=   30 tokens | Messages=19 | Cumulative In=  43,943

Turn 11: Input=  7,533 tokens | Output=  314 tokens | Messages=21 | Cumulative In=  51,476

Turn 12: Input=  8,404 tokens | Output=  365 tokens | Messages=23 | Cumulative In=  59,880

Turn 13: Input=  9,019 tokens | Output=   30 tokens | Messages=25 | Cumulative In=  68,899

Turn 14: Input=  9,238 tokens | Output=  342 tokens | Messages=27 | Cumulative In=  78,137

Turn 15: Input=  9,910 tokens | Output=  413 tokens | Messages=29 | Cumulative In=  88,047

Turn 16: Input= 10,580 tokens | Output=   30 tokens | Messages=31 | Cumulative In=  98,627

Turn 17: Input= 10,642 tokens | Output=  454 tokens | Messages=33 | Cumulative In= 109,269

============================================================
BASELINE RESULTS (NO COMPACTION)
============================================================
Total turns:   17
Input tokens:  109,269
Output tokens: 4,238
Total tokens:  113,507
============================================================

Now that we have our baseline, we have a better picture of how context grows without compaction. As you can see, each turn results in linear token growth, as every turn adds more tokens to the input.

This leads to high token consumption and potential context limits being reached quickly: the cumulative input climbs past a hundred thousand tokens for just 5 tickets.

Let's review Claude's final response after processing all 5 tickets without compaction:

print("".join(block.text for block in message.content if block.type == "text"))
I processed all 5 tickets, and the queue is now empty. Each one was classified, prioritized, routed, given a draft response and marked complete.

| Ticket | Category | Priority | Team | Notes |
|---|---|---|---|---|
| TICKET-1 (Morgan Jones) | billing | medium | billing-team | The charge date (10-09) doesn't match the signup date (10-04), so I flagged it for billing to review. |
| TICKET-2 (Jane Smith) | technical | medium | tech-support | The subject line says upload error, but the description is about device sync. I drafted a response for the sync problem. |
| TICKET-3 (Alex Johnson) | account | high | account-services | The email in the message differs from the ticket's contact email, so the draft asks the customer to confirm their registered address. |
| TICKET-4 (Morgan Davis) | account | high | account-services | The customer lost their phone and their backup codes aren't being accepted, so they're locked out. The draft asks for identity verification. |
| TICKET-5 (Sam Williams) | shipping | medium | logistics-team | The knowledge base has no shipping category. The draft asks for photos and promises no replacement timeline or policy. |

On three tickets the subject line and the description didn't match (TICKET-1, 2 and 4). I went by the description each time. You may want to check those three.

Understanding the Problem

In the baseline workflow above, Claude had to:

  • Process 5 support tickets sequentially
  • Complete 7 steps per ticket (fetch, classify, research, prioritize, route, draft, complete)
  • Make 35 tool calls with results accumulating in conversation history
  • Store every classification, every knowledge base search, every drafted response in memory

Why This Happens:

  1. Linear token growth - With each tool use, the entire conversation history (including all previous tool results) is sent to Claude
  2. Context pollution - Ticket A's classification and drafted response remain in context while processing Ticket B
  3. Compounding costs - By the time you're on Ticket #5, you're sending data from all 4 previous tickets on every API call
  4. Slower responses - Processing massive contexts takes longer
  5. Risk of hitting limits - Eventually you hit the context window limit

What We Actually Need: After completing Ticket A, we only need a brief summary (ticket resolved, category, priority) - not the full classification result, knowledge base search, and complete drafted response. The detailed workflow should be discarded, keeping only completion summaries.

Let's see how automatic context compaction solves this problem.

Enabling Automatic Context Compaction

Let's run the exact same customer service workflow, but with context compaction enabled. Two changes to the tool runner loop:

  • Pass betas=["compact-2026-09-04"] so the runner may send compaction requests
  • After each turn, compare message.usage.input_tokens with a threshold and call runner.compact_before_next_turn() when it is exceeded. The call only schedules the compaction: the runner finishes the current turn's tool calls first, then requests the summary. It accepts an optional config, {"type": "summarize", "instructions": "..."}, to replace the server's default summarization prompt (see Custom Summary Instructions below)

The compaction turn is yielded by the runner like any other message, with stop_reason == "compaction" and a single compaction content block holding the summary. Its token usage is reported under usage.iterations rather than the top-level counters, which the helper below accounts for.

For this customer service workflow, we'll use a 5,000 token threshold. This means after processing several tickets compaction will trigger. This allows Claude to:

  1. Keep completion summaries (tickets resolved, categories, outcomes)
  2. Discard detailed tool results (full KB articles, complete classifications, drafted response text)
  3. Start fresh when processing the next batch of tickets

This mimics how a real support agent works: resolve the ticket, document it briefly, move to the next case.

# Re-initialize queue and run with compaction
initialize_ticket_queue(num_tickets)
COMPACTION_THRESHOLD = 5000 # input tokens; low on purpose for the demo
def turn_tokens(message) -> tuple[int, int]:
"""Input/output tokens for one runner turn.
A compaction turn reports zero top-level tokens and lists the summarization
call under usage.iterations, so sum those instead.
"""
if message.stop_reason == "compaction" and message.usage.iterations:
return (
sum(it.input_tokens for it in message.usage.iterations),
sum(it.output_tokens for it in message.usage.iterations),
)
return message.usage.input_tokens, message.usage.output_tokens
total_input_compact = 0
total_output_compact = 0
turn_count_compact = 0
compaction_count = 0
runner = client.beta.messages.tool_runner(
model=MODEL,
max_tokens=4096,
tools=tools,
messages=messages,
betas=["compact-2026-09-04"],
)
for message in runner:
turn_count_compact += 1
input_tokens, output_tokens = turn_tokens(message)
total_input_compact += input_tokens
total_output_compact += output_tokens
messages_list = list(runner._params["messages"])
if message.stop_reason == "compaction":
# Before its next request the runner replaces the whole history with this single block.
compaction_count += 1
summary = next(block.content for block in message.content if block.type == "compaction")
print(f"\n{'=' * 60}")
print(
f"🔄 Compaction occurred! {len(messages_list)} messages condensed into one summary block"
)
print(" Summary after compaction:")
print(summary)
print(f"\n{'=' * 60}")
elif message.stop_reason == "tool_use" and input_tokens > COMPACTION_THRESHOLD:
# Schedule a compaction; it runs once this turn's tool calls are answered.
# (Only on tool_use turns: the final end_turn answer needs no compaction.)
runner.compact_before_next_turn()
print(
f"Turn {turn_count_compact:2d}: Input={input_tokens:7,} tokens | "
f"Output={output_tokens:5,} tokens | "
f"Messages={len(messages_list):2d} | "
f"Cumulative In={total_input_compact:8,}"
)
print(f"\n{'=' * 60}")
print("OPTIMIZED RESULTS (WITH COMPACTION)")
print(f"{'=' * 60}")
print(f"Total turns: {turn_count_compact}")
print(f"Compactions: {compaction_count}")
print(f"Input tokens: {total_input_compact:,}")
print(f"Output tokens: {total_output_compact:,}")
print(f"Total tokens: {total_input_compact + total_output_compact:,}")
print(f"{'=' * 60}")
Turn  1: Input=  1,732 tokens | Output=   30 tokens | Messages= 1 | Cumulative In=   1,732

Turn  2: Input=  1,955 tokens | Output=  317 tokens | Messages= 3 | Cumulative In=   3,687

Turn  3: Input=  2,602 tokens | Output=  101 tokens | Messages= 5 | Cumulative In=   6,289

Turn  4: Input=  3,233 tokens | Output=  317 tokens | Messages= 7 | Cumulative In=   9,522

Turn  5: Input=  3,803 tokens | Output=   30 tokens | Messages= 9 | Cumulative In=  13,325

Turn  6: Input=  4,033 tokens | Output=  392 tokens | Messages=11 | Cumulative In=  17,358

Turn  7: Input=  5,026 tokens | Output=  321 tokens | Messages=13 | Cumulative In=  22,384


============================================================
🔄 Compaction occurred! 15 messages condensed into one summary block
   Summary after compaction:
## Progress summary

**Task**: Process 5 support tickets one at a time. For each ticket: get_next_ticket → classify_ticket → search_knowledge_base → set_priority → route_to_team → draft_response → mark_complete. Keep going until the queue is empty (the 5 tickets are expected in total).

**Completed so far (2 of 5):**
- **TICKET-1** (John Williams, tracking number 1Z856942470332 reported invalid): classified shipping, priority medium, routed to logistics-team. The draft response made no unsupported claims and said the logistics team would verify the number. Marked complete.
- **TICKET-2** (Alex Jones, order #ORD-66101: received Yellow Widget Lite instead of Green Device Max; the subject line said "change delivery address", which doesn't match the description): classified shipping, priority medium, routed to logistics-team. The draft apologized, said logistics would arrange the correct item and return instructions, and offered to pass on a new address if needed. Marked complete.

**Notes on the tools:**
- The KB only has the categories billing, technical, and account. "shipping" returns an error. For shipping tickets, I searched the nearest available category (billing for return/refund topics, technical for tracking). Neither had shipping-specific content, so the drafts avoided inventing policy details.
- KB content seen: the billing refund policy (refunds take 5-7 business days for valid cancellations, pro-rated for annual plans), payment methods (Visa, Mastercard, Amex, PayPal; update them in Account Settings), and the billing cycle. Technical covers common errors, system requirements (4GB RAM, Chrome 90+/Firefox 88+/Safari 14+), and troubleshooting (clear cache, incognito, enable JavaScript).
- Valid teams: billing-team, tech-support, account-services, product-success, logistics-team.
- Independent calls (classify, search, priority, route) can be made in parallel. Draft and mark_complete come after them.

**Next step**: Call get_next_ticket for TICKET-3, then repeat the full workflow for TICKET-3, 4, and 5. Stop when the queue returns an empty error. Then give a brief final summary of all processed tickets.

============================================================
Turn  8: Input=  5,661 tokens | Output=  758 tokens | Messages=15 | Cumulative In=  28,045

Turn  9: Input=  2,087 tokens | Output=   30 tokens | Messages= 1 | Cumulative In=  30,132

Turn 10: Input=  2,303 tokens | Output=  355 tokens | Messages= 3 | Cumulative In=  32,435

Turn 11: Input=  3,145 tokens | Output=  355 tokens | Messages= 5 | Cumulative In=  35,580

Turn 12: Input=  3,751 tokens | Output=   30 tokens | Messages= 7 | Cumulative In=  39,331

Turn 13: Input=  3,982 tokens | Output=  314 tokens | Messages= 9 | Cumulative In=  43,313

Turn 14: Input=  4,890 tokens | Output=  426 tokens | Messages=11 | Cumulative In=  48,203

Turn 15: Input=  5,569 tokens | Output=   30 tokens | Messages=13 | Cumulative In=  53,772


============================================================
🔄 Compaction occurred! 15 messages condensed into one summary block
   Summary after compaction:
## Progress summary

**Task**: Process 5 support tickets one at a time. For each ticket: get_next_ticket → classify_ticket → search_knowledge_base → set_priority → route_to_team → draft_response → mark_complete. Keep going until the queue is empty (the 5 tickets are expected in total).

**Completed so far (4 of 5):**
- **TICKET-1** (John Williams, tracking number reported invalid): shipping, medium, logistics-team. Draft made no unsupported claims. Marked complete.
- **TICKET-2** (Alex Jones, wrong item received on order #ORD-66101; the subject line didn't match the description): shipping, medium, logistics-team. Marked complete.
- **TICKET-3** (Alex Jones, password reset emails not arriving): account, medium, account-services. The draft cited the KB (emails come from noreply@support.example.com, the link expires in 1 hour). It flagged that the ticket email (customer8695@) differs from the email in the description (customer6750@) and asked the customer to confirm which is registered. Marked complete.
- **TICKET-4** (John Smith, unrecognized $49.99 charge on 2026-10-20): billing, high, billing-team. The draft explained the billing cycle, did not claim to know what the charge was, mentioned refund policy conditionally, and asked for the last four digits only. Marked complete.

**Currently in progress: TICKET-5** (just fetched, no actions taken yet):
- Jane Brown, customer4919@example.com, subject "Account locked after failed login". She says the account was locked after too many wrong password attempts. She now remembers her password, but the unlock email link isn't working. Pre-set category: account. Created 2026-10-19.
- Plan: classify as account, search the account KB (e.g. "account locked unlock"), set priority (medium, or high since she is locked out), route to account-services. Then draft a response that doesn't invent unlock policy beyond what the KB returns, and mark complete. The KB has password_reset (emails from noreply@support.example.com, check spam, link expires in 1 hour), 2fa_backup, and email_change, so there may be no lockout-specific entry. If so, say the Account Services team will verify her identity and help unlock the account, and suggest requesting a new link since the old one may have expired.

**Notes on the tools:**
- The KB only has the categories billing, technical, and account. "shipping" returns an error.
- Valid teams: billing-team, tech-support, account-services, product-success, logistics-team.
- Independent calls (classify, search, priority, route) can be made in parallel. Draft and mark_complete come after them (these two can also go in parallel).

**Next steps**: Finish TICKET-5 with the full workflow. Then call get_next_ticket once more to confirm the queue is empty. Then give a brief final summary of all 5 processed tickets, noting the oddities flagged: the TICKET-2 subject/description mismatch, the TICKET-3 email mismatch, and the absence of a shipping KB category.

============================================================
Turn 16: Input=  5,840 tokens | Output=1,027 tokens | Messages=15 | Cumulative In=  59,612

Turn 17: Input=  2,356 tokens | Output=  309 tokens | Messages= 1 | Cumulative In=  61,968

Turn 18: Input=  3,217 tokens | Output=  386 tokens | Messages= 3 | Cumulative In=  65,185

Turn 19: Input=  3,910 tokens | Output=  460 tokens | Messages= 5 | Cumulative In=  69,095

============================================================
OPTIMIZED RESULTS (WITH COMPACTION)
============================================================
Total turns:   19
Compactions:   2
Input tokens:  69,095
Output tokens: 5,988
Total tokens:  75,083
============================================================

With context compaction enabled, token usage per turn no longer grows linearly: it drops back after each compaction event, and the following turns start from the summary instead of the full history.

Compared to the baseline, the run uses far fewer input tokens (the comparison table below has the totals). We've also printed the summary returned by each compaction, showing how prior ticket details were condensed.

Let's look at the final response after processing all 5 tickets with compaction enabled.

print("".join(block.text for block in message.content if block.type == "text"))
All 5 tickets are processed and the queue is empty.

| Ticket | Customer | Issue | Category | Priority | Team |
|---|---|---|---|---|---|
| TICKET-1 | John Williams | Tracking number reported invalid | shipping | medium | logistics-team |
| TICKET-2 | Alex Jones | Wrong item received on order #ORD-66101 | shipping | medium | logistics-team |
| TICKET-3 | Alex Jones | Password reset emails not arriving | account | medium | account-services |
| TICKET-4 | John Smith | Unrecognized $49.99 charge | billing | high | billing-team |
| TICKET-5 | Jane Brown | Account locked, unlock link not working | account | high | account-services |

Each ticket has a draft response and is marked complete.

Three things to check:
- **TICKET-2:** The subject line doesn't match the description. I went by the description.
- **TICKET-3:** The email on the ticket (customer8695@) differs from the one in the description (customer6750@). The draft asks the customer to confirm which one is registered.
- **Knowledge base gaps:** The knowledge base has no shipping category, so the shipping drafts make no policy claims. It also has no entry on account lockouts. The TICKET-5 draft only cites the password-reset link expiry and hands the unlock to Account Services.

Comparing Results

With compaction enabled, we can see a clear difference between the two runs in token savings, while preserving the quality of the workflow and final summary.

Here's what changed with automatic context compaction:

  1. Context resets after several tickets - When a few tickets' worth of tool results pushes a turn past 5k input tokens, the loop calls compact_before_next_turn() and the runner:

    • Finishes the current turn's tool calls, then sends a compaction request
    • Receives a server-generated summary as a compaction block
    • Replaces the conversation history (detailed classifications, KB searches, responses) with that block
    • Continues with only the summary
  2. Input tokens stay bounded - Instead of accumulating to 100k+ as we process more tickets, input tokens reset after each compaction. When processing Ticket #5, we're NOT carrying the full tool results from Tickets #1-4.

  3. Task completes successfully - The workflow continues smoothly through all tickets without hitting context limits

  4. Quality is preserved - The summaries retain critical information:

    • Tickets processed with their IDs
    • Categories and priorities assigned
    • Teams routed to
    • Overall progress status

    All tickets are still properly classified, prioritized, routed, and responded to.

  5. Natural workflow - This mirrors how real support agents work: resolve a ticket, document it briefly in the system, close it, move to the next one. You don't keep every knowledge base article and full response draft open while working on new tickets.

Let's visualize the token savings:

# Compare baseline vs compaction
print("=" * 70)
print("TOKEN USAGE COMPARISON")
print("=" * 70)
print(f"{'Metric':<30} {'Baseline':<20} {'With Compaction':<20}")
print("-" * 70)
print(f"{'Input tokens:':<30} {total_input:>19,} {total_input_compact:>19,}")
print(f"{'Output tokens:':<30} {total_output:>19,} {total_output_compact:>19,}")
print(
f"{'Total tokens:':<30} {total_input + total_output:>19,} {total_input_compact + total_output_compact:>19,}"
)
print(f"{'Compactions:':<30} {'N/A':>19} {compaction_count:>19}")
print("=" * 70)
# Calculate savings
token_savings = (total_input + total_output) - (total_input_compact + total_output_compact)
savings_percent = (
(token_savings / (total_input + total_output)) * 100 if (total_input + total_output) > 0 else 0
)
print(f"\n💰 Token Savings: {token_savings:,} tokens ({savings_percent:.1f}% reduction)")
======================================================================
TOKEN USAGE COMPARISON
======================================================================
Metric                         Baseline             With Compaction     
----------------------------------------------------------------------
Input tokens:                              109,269              69,095
Output tokens:                               4,238               5,988
Total tokens:                              113,507              75,083
Compactions:                                   N/A                   2
======================================================================

💰 Token Savings: 38,424 tokens (33.9% reduction)

How Compaction Works Under the Hood

When you call runner.compact_before_next_turn(), the tool runner:

  1. Lets the current turn finish - pending tool calls are run and their results appended first, because the API can't compact a conversation that ends with an unanswered tool call
  2. Sends a compaction request - the same model, system prompt, tools and messages, plus compaction={"type": "summarize"}. Nothing is sampled after the summary, so this request returns only the summary
  3. Receives a compaction block - the API summarizes the conversation server-side (completed tickets, progress, what to do next) and returns it as a single content block with stop_reason == "compaction"
  4. Replaces history - the runner swaps the entire message list (including all tool results) for that one block, which must be sent back first and unmodified
  5. Resumes processing - the next request continues from the compressed context, and Claude picks up the next ticket

Customizing Compaction Configuration

You can customize how compaction works to fit your specific use case. Here are the key knobs:

Adjusting the Threshold

With runner-driven compaction the threshold is your own loop logic, so it can be any value or any condition (for example, "after every third mark_complete call"):

COMPACTION_THRESHOLD = 5000 # Compact after processing a few tickets
for message in runner:
if message.stop_reason == "tool_use" and message.usage.input_tokens > COMPACTION_THRESHOLD:
runner.compact_before_next_turn()

The threshold should not be set too low, otherwise the summary itself could trigger a compaction. We set a threshold of 5,000 tokens for demonstration purposes, but in practice, experiment with different settings to find what works best for your workflow.

Here some general guidelines:

  • Low thresholds (5k-20k):

    • Use for iterative task processing with clear boundaries
    • More frequent compaction, minimal context accumulation
    • Best for sequential entity processing
  • Medium thresholds (50k-100k):

    • Multi-phase workflows with fewer, larger natural checkpoints
    • Balance between context retention and management
    • Suitable for workflows with expensive tool calls
  • High thresholds (100k-150k):

    • Tasks requiring substantial historical context
    • Less frequent compaction preserves more raw details
    • Higher per-call costs but fewer compactions

If all you need is a fixed token trigger of 50k or more, you don't need loop logic at all: pass a context_management compaction edit and the API compacts automatically (see the compaction docs). The runner refuses to combine that edit with compact_before_next_turn(), so pick one approach per runner.

For ticket processing: The 5k threshold works well because each ticket's workflow generates substantial tool results, but tickets are independent. After resolving Ticket A, you don't need its detailed KB searches when processing Ticket B.

Custom Summary Instructions

You can pass instructions to replace the server's default summarization prompt. This is especially useful for customer service workflows where you need to preserve specific types of information.

For example, we could define a custom prompt based on our requirements:

  • Ticket summaries for all completed tickets
  • Categories and priorities assigned
  • Teams routed to
  • Progress status (tickets completed, tickets remaining)
  • Next steps in the workflow
runner.compact_before_next_turn(
{
"type": "summarize",
"instructions": """You are processing customer support tickets from a queue.
Create a focused summary that preserves:
1. **COMPLETED TICKETS**: For each ticket you've fully processed:
- Ticket ID and customer name
- Issue category and priority assigned
- Team routed to
- Brief outcome
2. **PROGRESS STATUS**:
- How many tickets you've completed
- Approximately how many remain in the queue
3. **NEXT STEPS**: Continue processing the next ticket
Format with clear sections.""",
}
)

Compaction Without Tools: Simple Chat Loop

While the examples above focus on tool-heavy agentic workflows, context compaction is also valuable for simple conversational applications where users drive the conversation.

Note: compact_before_next_turn() is a tool-runner method. In a plain chat loop you can either send compaction={"type": "summarize"} on a client.beta.messages.create() call yourself and put the returned block first in messages, or implement the summarization manually as shown below, which works with any model and gives you full control over the summary prompt.

Consider a chat application where users are having extended conversations with Claude—discussing complex topics, iterating on ideas, or working through problems. As the conversation grows, you face the same context accumulation challenges.

The Difference: Instead of tool use triggering token growth, it's the back-and-forth conversation itself. Each exchange adds messages to the history:

  • User asks a question
  • Claude provides a detailed response
  • User asks for clarification or elaboration
  • Claude responds with more context
  • This repeats dozens or hundreds of times

Without compaction, by turn 50 you're sending the entire conversation history (all 50 exchanges) on every API call.

The Solution: Implement compaction manually in your chat loop using the same pattern:

  1. Track token usage after each turn
  2. When threshold is exceeded, request a summary
  3. Replace conversation history with the summary
  4. Continue the conversation with compressed context

Let's see how to implement this:

#!/usr/bin/env python3
"""
Simple Compaction Example - User-Driven Chat Loop
This shows the basic pattern for a chat application with compaction.
No tools required - just a simple loop where the user drives continuation.
"""
# Configuration
COMPACTION_THRESHOLD = 3000 # Compact when tokens exceed this (low for demo purposes)
# Structured summarization prompt for compaction
SUMMARY_PROMPT = """You have been working on the task described above but have not yet completed it. Write a continuation summary that will allow you (or another instance of yourself) to resume work efficiently in a future context window where the conversation history will be replaced with this summary. Your summary should be structured, concise, and actionable. Include:
1. **Task Overview**
- The user's core request and success criteria
- Any clarifications or constraints they specified
2. **Current State**
- What has been completed so far
- Files created, modified, or analyzed (with paths if relevant)
- Key outputs or artifacts produced
3. **Important Discoveries**
- Technical constraints or requirements uncovered
- Decisions made and their rationale
- Errors encountered and how they were resolved
- What approaches were tried that didn't work (and why)
4. **Next Steps**
- Specific actions needed to complete the task
- Any blockers or open questions to resolve
- Priority order if multiple steps remain
5. **Context to Preserve**
- User preferences or style requirements
- Domain-specific details that aren't obvious
- Any promises made to the user
Be concise but complete—err on the side of including information that would prevent duplicate work or repeated mistakes.
Write in a way that enables immediate resumption of the task.
Wrap your summary in <summary></summary> tags."""
# Message history
messages = []
print("Chat with Claude (type 'quit' to exit, or just hit Enter to continue)")
print("This is a demonstration - try having a conversation and watch compaction trigger")
print("=" * 60)
# Simulate a conversation for demo purposes
demo_messages = [
"Help me understand how Python decorators work",
"Can you show me an example with a timing decorator?",
"How would I make a decorator that takes arguments?",
]
for user_input in demo_messages:
print(f"\nYou: {user_input}")
# Add user message
messages.append({"role": "user", "content": user_input})
# Get Claude's response
response = client.messages.create(
model=MODEL,
max_tokens=2048,
messages=messages,
)
messages.append(
{
"role": "assistant",
"content": response.content,
}
)
print("\nClaude: ", end="")
for block in response.content:
if block.type == "text":
print(f"{block.text[:300]} ...")
# Check if we should compact
usage = response.usage
# Calculate total tokens (includes cache tokens)
total_input_tokens = (
usage.input_tokens
+ (usage.cache_creation_input_tokens or 0)
+ (usage.cache_read_input_tokens or 0)
)
total_tokens = total_input_tokens + usage.output_tokens
cache_info = ""
if usage.cache_creation_input_tokens or usage.cache_read_input_tokens:
cache_info = f" (cache: {usage.cache_creation_input_tokens or 0} write + {usage.cache_read_input_tokens or 0} read)"
print(
f"\n[Tokens: {total_input_tokens} in{cache_info} + {usage.output_tokens} out = {total_tokens} total]"
)
if total_tokens > COMPACTION_THRESHOLD:
print(f"\n{'=' * 60}")
print(f"🔄 Compacting conversation... {len(messages)} messages → ", end="", flush=True)
# Get summary using structured prompt
summary_response = client.messages.create(
model=MODEL,
max_tokens=4096,
messages=messages + [{"role": "user", "content": SUMMARY_PROMPT}],
)
summary_text = "".join(
block.text for block in summary_response.content if block.type == "text"
)
# Replace history with summary
messages = [{"role": "user", "content": summary_text}]
print("1 message")
print(f"{'=' * 60}\n")
print(f"Final conversation messages: {messages[-1].get('content')}")
print("\nDemo complete! In a real application, this loop would continue with user input.")
Chat with Claude (type 'quit' to exit, or just hit Enter to continue)
This is a demonstration - try having a conversation and watch compaction trigger
============================================================

You: Help me understand how Python decorators work


Claude: # Python Decorators

A decorator is a function that takes another function and returns a modified or enhanced version of it, without you changing the original function's code.

## 1. The building blocks

Decorators rely on two Python features.

**Functions are objects.** You can assign them to varia ...

[Tokens: 20 in + 1497 out = 1517 total]

You: Can you show me an example with a timing decorator?


Claude: # Timing Decorator: Full Example

## Basic version

```python
import time
import functools

def timer(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        p ...

[Tokens: 1536 in + 1437 out = 2973 total]

You: How would I make a decorator that takes arguments?


Claude: # Decorators That Take Arguments

## The core idea

A normal decorator is a function that takes a function. To pass arguments, add one more layer: a function that takes the arguments and **returns** a decorator.

```
decorator_factory(args)  →  decorator(func)  →  wrapper(*args, **kwargs)
```

## Th ...

[Tokens: 2992 in + 1641 out = 4633 total]

============================================================
🔄 Compacting conversation... 6 messages → 
1 message
============================================================

Final conversation messages: <summary>
## Continuation Summary

### 1. Task Overview
The user is learning Python decorators through a conversational Q&A. There is no fixed deliverable or file output. The goal is clear, accurate, example-driven explanations. No explicit constraints were given beyond the questions asked. The task is not "unfinished" in a strict sense; the conversation is open-ended and the user may ask further follow-ups.

### 2. Current State
Three user questions have been answered, all in chat (no files created or modified):

1. **"Help me understand how Python decorators work"** – Covered: functions as first-class objects, nested/returned functions, manual decoration (`greet = shout(greet)`), `@` syntax as sugar, `*args/**kwargs` wrappers, `functools.wraps`, decorators with arguments (brief), a timing example, stacking order (`@a @b def f` = `a(b(f))`), a table of common built-in decorators (`staticmethod`, `classmethod`, `property`, `lru_cache`, `dataclass`), and a mental model. Ended by offering to go deeper on class-based decorators, decorating classes, and closures.
2. **"Can you show me an example with a timing decorator?"** – Provided: a basic `timer` using `time.perf_counter()`, usage with args/return values, a `try/finally` version so timing prints even on exceptions, a configurable version `timer(label=None, log=False)` (requires parentheses), an `async_timer` for coroutines, a step-by-step call flow, and a note pointing to `timeit` for serious benchmarking.
3. **"How would I make a decorator that takes arguments?"** – Provided: the three-layer pattern (factory → decorator → wrapper), the desugaring `@repeat(3)` = `repeat(3)(func)`, closures as the reason inner layers see outer args, a `require_role` access-control example, an optional-parentheses pattern (`func=None, *, times=2`), common mistakes (missing parentheses, missing returns, omitting `wraps`), and a class-based alternative (`__init__` stores args, `__call__` takes the function).

### 3. Important Discoveries / Decisions
- Consistent teaching approach: build from first principles, show the desugared equivalent, then give practical examples.
- Always include `@functools.wraps(func)` in examples, and explain why.
- Use `time.perf_counter()` over `time.time()` for timing.
- Use `try/finally` in timing decorators so exceptions don't skip the measurement.
- Async functions need an `async def wrapper` with `await`; a sync wrapper would only time coroutine creation.
- No errors or failed approaches have occurred.

### 4. Next Steps
- No required actions remain. Await the user's next question.
- Likely follow-ups, already offered or naturally adjacent: class-based decorators (a class with `__call__` that wraps a function, and keeping state such as call counters), decorating classes, closures in depth, `lru_cache`/memoization, decorators on methods (handling `self`), stacking order subtleties, a retry/caching/validation decorator example, type hints for decorators (`ParamSpec`, `TypeVar`), and testing decorated functions (`__wrapped__`).

### 5. Context to Preserve
- Style: Markdown with headers, short code blocks with printed output shown in comments or output blocks, brief explanatory prose, and a concise summary or mental model at the end. Code is runnable and minimal.
- The user appears to be a learner, so build up concepts incrementally and avoid unexplained jargon.
- Promises made: offered to go deeper on class-based decorators, decorating classes, and how closures work, if the user wants.
</summary>

Demo complete! In a real application, this loop would continue with user input.

Understanding the Chat Loop Pattern

The example above demonstrates manual compaction in a conversational context. Here's how it works:

Key Components:

  1. Token Tracking: After each response, calculate total tokens (input + output + cache tokens)
  2. Threshold Check: When total exceeds threshold, trigger compaction
  3. Summary Request: Send the same structured SUMMARY_PROMPT to Claude
  4. History Replacement: Replace entire message history with just the summary
  5. Continue: Next user message builds on the summary, not full history

When to Use This Pattern:

  • Extended brainstorming sessions: Users exploring ideas with Claude over many turns
  • Learning conversations: Tutorials or explanations that span dozens of exchanges
  • Iterative refinement: Users providing feedback on drafts, designs, or solutions
  • Chat applications: Any multi-turn conversation interface

Key Differences from Tool Runner:

AspectTool Runner (compact_before_next_turn)Chat Loop (Manual)
TriggerYour threshold check schedules it; runner picks the momentYou implement threshold check
SummaryGenerated server-side, returned as a compaction blockYou make explicit API call with your own prompt
History ManagementRunner replaces messagesYou manually replace list
Use CaseAgentic workflows with toolsUser-driven conversations

Production Considerations:

  1. Adjust threshold: Use larger thresholds for real applications
  2. Customize summary prompt: Tailor to your conversation type (brainstorming vs. technical support vs. tutoring)
  3. Show user indicators: Display a message like "Summarizing conversation..." so users understand the pause
  4. Preserve key context: Ensure the summary prompt captures domain-specific information your users care about

This pattern gives you full control over when and how compaction happens, making it ideal for conversational applications where you aren't using the tool runner.

Limitations and Considerations

While automatic context compaction is powerful, there are important limitations to understand:

Measuring the Threshold

Watch what you count: this cookbook triggers on usage.input_tokens. With prompt caching enabled most of the history shows up under cache_read_input_tokens instead, and server-side tool loops (web search, code execution) report several usage.iterations per turn. Sum the counters that reflect real history size in your workflow, or the trigger fires too late or too early.

Timing: a scheduled compaction runs only after the current turn's tool calls have been answered, and it is skipped (with a warning) if the run ends on a turn with unanswered tool calls.

Information Loss

Trade-off: Summaries inherently lose some information. While Claude is good at identifying key points, some details will be compressed or omitted.

In ticket processing:

  • ✅ Retained: Ticket IDs, categories, priorities, teams, outcomes, progress status
  • ❌ Lost: Full knowledge base article text, complete drafted response text, detailed classification reasoning

This is usually acceptable, you don't need every KB article and full response text in perpetuity, just the completion records.

Mitigation:

  • Use custom summary prompts to preserve critical information
  • Set higher thresholds for tasks requiring extensive historical context
  • Structure your tasks to be modular (each phase builds on summaries, not raw details)

When NOT to Use Compaction

Avoid compaction for:

  1. Short tasks: If your task completes within 50k-100k tokens, compaction adds unnecessary overhead
  2. Tasks requiring full audit trails: Some tasks need access to ALL previous details
  3. Highly iterative refinement: Tasks where each step critically depends on exact details from all previous steps

When TO Use Compaction

Compaction is ideal for:

  1. Sequential processing: Like our ticket workflow—process multiple items one after another
  2. Multi-phase workflows: Where each phase can summarize progress before moving on
  3. Iterative data processing: Processing large datasets in chunks or entities one at a time
  4. Extended analysis sessions: Analyzing data across many entities
  5. Batch operations: Processing hundreds of items where each is independent

Ticket processing is a perfect use case because:

  • Each ticket workflow is largely independent
  • You need completion summaries, not full tool results
  • Natural compaction points exist (after completing several tickets)
  • The workflow is iterative and sequential

Summary

Automatic context compaction is a powerful feature that enables long-running agentic workflows to exceed typical context limits. In this cookbook, we've explored compaction through a customer service ticket processing workflow.

Next Steps

Try implementing compaction in your own workflows:

  1. Identify natural compaction points (after processing each item, completing each phase, etc.)
  2. Start with an aggressive threshold (5k-10k) if you have clear per-item boundaries
  3. Use custom summary prompts to preserve critical information
  4. Monitor when compaction triggers and verify quality is maintained
  5. Adjust threshold based on your specific needs

For more on effective context management, see Effective Context Engineering for AI Agents.

Was this page helpful?