Automatic Context Compaction
Long-running agentic tasks can often exceed context limits. Tool heavy workflows or long conversations quickly consume the token context window. In Effective Context Engineering for AI Agents(opens in new tab), we discussed how managing context can help avoid performance degradation and context rot.
The Anthropic Python SDK's tool runner can help manage this context: when your loop decides the conversation has grown too large, one call to runner.compact_before_next_turn() has the API summarize the history server-side and the runner carries on from that summary, allowing tasks to continue beyond the model's context window limit.
In this cookbook, we'll demonstrate context compaction through an agentic customer service workflow. Imagine you've built an AI customer service agent tasked with processing a queue of support tickets. For each ticket, you must classify the issue, search the knowledge base, set priority, route to the appropriate team, draft a response, and mark it complete. As you process ticket after ticket, the conversation history fills with classifications, knowledge base searches, and drafted responses—quickly consuming thousands of tokens.
What is Context Compaction?
When building agentic workflows with tool use, conversations can grow very large as the agent iterates on complex tasks. Runner-driven compaction manages this in four steps:
- Your loop monitors token usage per turn in the conversation
- When your threshold is exceeded, you call
runner.compact_before_next_turn() - Once the current turn (including its tool calls) finishes, the runner sends a compaction request. The API summarizes the conversation server-side and returns a single
compactionblock (stop_reason == "compaction") - The runner replaces the conversation history with that block and continues the task with the compressed context
By the end of this cookbook, you'll be able to:
- Understand how to effectively manage context limits in iterative workflows
- Write agents that leverage automatic context compaction
- Design workflows that maintain focus across multiple iterations
Prerequisites
Before following this guide, ensure you have:
Required Knowledge
- Basic understanding of agentic patterns and tool calling
Required Tools
- Python 3.11 or higher
- Anthropic API key
- Anthropic SDK >= 2.0.0 (uses the
compact-2026-09-04beta)
Just want compaction to happen automatically? Server-side compaction(opens in new tab) via
context_managementcompacts on its own once input tokens cross a trigger you set (50k tokens or more), with no loop logic at all.This cookbook covers runner-driven compaction, where your code decides when to compact: useful for low thresholds, natural item boundaries (compact after every N tickets), or custom summarization instructions.
Setup
First, install the required dependencies:
Note: Ensure your .env file contains:
ANTHROPIC_API_KEY=your_key_here
Load your environment variables and configure the client. We also load a helper utility to visualize Claude message responses.
Setting the Stage
In utils/customer_service_tools.py(opens in new tab), we've defined several functions for processing customer support tickets:
get_next_ticket()- Retrieves the next unprocessed ticket from the queueclassify_ticket(ticket_id, category)- Categorizes issues as billing, technical, account, product, or shippingsearch_knowledge_base(query)- Finds relevant help articles and solutionsset_priority(ticket_id, priority)- Assigns priority levels (low, medium, high, urgent)route_to_team(ticket_id, team)- Routes tickets to the appropriate support teamdraft_response(ticket_id, response_text)- Creates customer-facing responsesmark_complete(ticket_id)- Finalizes processed tickets
For a customer service agent, these tools enable processing tickets systematically. Each ticket requires classification, research, prioritization, routing, and response drafting. When processing 20-30 tickets in sequence, the conversation history fills with tool results from every classification, every knowledge base search, and every drafted response, causing linear token growth.
The beta_tool decorator is used on the tools to make them accessible to the Claude agent. The decorator extracts the function arguments and docstring and provides these to Claude as tool metadata.
Baseline: Running Without Compaction
Let's start with a realistic customer service scenario: Processing a queue of support tickets.
The workflow looks like this:
For Each Ticket:
- Fetch the ticket using
get_next_ticket() - Classify the issue category (billing, technical, account, product, shipping)
- Search the knowledge base for relevant information
- Set appropriate priority (low, medium, high, urgent)
- Route to the correct team
- Draft a customer response
- Mark the ticket complete
- Move to the next ticket
The Challenge: With 5 tickets in the queue, and each requiring 7 tool calls, Claude will make 35 or more tool calls. The results from each step including classification knowledge base search, and drafted responses accumulate in the conversation history. Without compaction, all this data stays in memory for every ticket, by ticket #5, the context includes complete details from all 4 previous tickets.
Let's run this workflow without compaction first and observe what happens:
Turn 1: Input= 1,732 tokens | Output= 30 tokens | Messages= 1 | Cumulative In= 1,732 Turn 2: Input= 1,964 tokens | Output= 318 tokens | Messages= 3 | Cumulative In= 3,696 Turn 3: Input= 2,881 tokens | Output= 378 tokens | Messages= 5 | Cumulative In= 6,577 Turn 4: Input= 3,516 tokens | Output= 30 tokens | Messages= 7 | Cumulative In= 10,093 Turn 5: Input= 3,744 tokens | Output= 313 tokens | Messages= 9 | Cumulative In= 13,837 Turn 6: Input= 4,875 tokens | Output= 409 tokens | Messages=11 | Cumulative In= 18,712 Turn 7: Input= 5,538 tokens | Output= 30 tokens | Messages=13 | Cumulative In= 24,250 Turn 8: Input= 5,758 tokens | Output= 309 tokens | Messages=15 | Cumulative In= 30,008 Turn 9: Input= 6,619 tokens | Output= 443 tokens | Messages=17 | Cumulative In= 36,627 Turn 10: Input= 7,316 tokens | Output= 30 tokens | Messages=19 | Cumulative In= 43,943 Turn 11: Input= 7,533 tokens | Output= 314 tokens | Messages=21 | Cumulative In= 51,476 Turn 12: Input= 8,404 tokens | Output= 365 tokens | Messages=23 | Cumulative In= 59,880 Turn 13: Input= 9,019 tokens | Output= 30 tokens | Messages=25 | Cumulative In= 68,899 Turn 14: Input= 9,238 tokens | Output= 342 tokens | Messages=27 | Cumulative In= 78,137 Turn 15: Input= 9,910 tokens | Output= 413 tokens | Messages=29 | Cumulative In= 88,047 Turn 16: Input= 10,580 tokens | Output= 30 tokens | Messages=31 | Cumulative In= 98,627 Turn 17: Input= 10,642 tokens | Output= 454 tokens | Messages=33 | Cumulative In= 109,269 ============================================================ BASELINE RESULTS (NO COMPACTION) ============================================================ Total turns: 17 Input tokens: 109,269 Output tokens: 4,238 Total tokens: 113,507 ============================================================
Now that we have our baseline, we have a better picture of how context grows without compaction. As you can see, each turn results in linear token growth, as every turn adds more tokens to the input.
This leads to high token consumption and potential context limits being reached quickly: the cumulative input climbs past a hundred thousand tokens for just 5 tickets.
Let's review Claude's final response after processing all 5 tickets without compaction:
I processed all 5 tickets, and the queue is now empty. Each one was classified, prioritized, routed, given a draft response and marked complete. | Ticket | Category | Priority | Team | Notes | |---|---|---|---|---| | TICKET-1 (Morgan Jones) | billing | medium | billing-team | The charge date (10-09) doesn't match the signup date (10-04), so I flagged it for billing to review. | | TICKET-2 (Jane Smith) | technical | medium | tech-support | The subject line says upload error, but the description is about device sync. I drafted a response for the sync problem. | | TICKET-3 (Alex Johnson) | account | high | account-services | The email in the message differs from the ticket's contact email, so the draft asks the customer to confirm their registered address. | | TICKET-4 (Morgan Davis) | account | high | account-services | The customer lost their phone and their backup codes aren't being accepted, so they're locked out. The draft asks for identity verification. | | TICKET-5 (Sam Williams) | shipping | medium | logistics-team | The knowledge base has no shipping category. The draft asks for photos and promises no replacement timeline or policy. | On three tickets the subject line and the description didn't match (TICKET-1, 2 and 4). I went by the description each time. You may want to check those three.
Understanding the Problem
In the baseline workflow above, Claude had to:
- Process 5 support tickets sequentially
- Complete 7 steps per ticket (fetch, classify, research, prioritize, route, draft, complete)
- Make 35 tool calls with results accumulating in conversation history
- Store every classification, every knowledge base search, every drafted response in memory
Why This Happens:
- Linear token growth - With each tool use, the entire conversation history (including all previous tool results) is sent to Claude
- Context pollution - Ticket A's classification and drafted response remain in context while processing Ticket B
- Compounding costs - By the time you're on Ticket #5, you're sending data from all 4 previous tickets on every API call
- Slower responses - Processing massive contexts takes longer
- Risk of hitting limits - Eventually you hit the context window limit
What We Actually Need: After completing Ticket A, we only need a brief summary (ticket resolved, category, priority) - not the full classification result, knowledge base search, and complete drafted response. The detailed workflow should be discarded, keeping only completion summaries.
Let's see how automatic context compaction solves this problem.
Enabling Automatic Context Compaction
Let's run the exact same customer service workflow, but with context compaction enabled. Two changes to the tool runner loop:
- Pass
betas=["compact-2026-09-04"]so the runner may send compaction requests - After each turn, compare
message.usage.input_tokenswith a threshold and callrunner.compact_before_next_turn()when it is exceeded. The call only schedules the compaction: the runner finishes the current turn's tool calls first, then requests the summary. It accepts an optional config,{"type": "summarize", "instructions": "..."}, to replace the server's default summarization prompt (see Custom Summary Instructions below)
The compaction turn is yielded by the runner like any other message, with stop_reason == "compaction" and a single compaction content block holding the summary. Its token usage is reported under usage.iterations rather than the top-level counters, which the helper below accounts for.
For this customer service workflow, we'll use a 5,000 token threshold. This means after processing several tickets compaction will trigger. This allows Claude to:
- Keep completion summaries (tickets resolved, categories, outcomes)
- Discard detailed tool results (full KB articles, complete classifications, drafted response text)
- Start fresh when processing the next batch of tickets
This mimics how a real support agent works: resolve the ticket, document it briefly, move to the next case.
Turn 1: Input= 1,732 tokens | Output= 30 tokens | Messages= 1 | Cumulative In= 1,732 Turn 2: Input= 1,955 tokens | Output= 317 tokens | Messages= 3 | Cumulative In= 3,687 Turn 3: Input= 2,602 tokens | Output= 101 tokens | Messages= 5 | Cumulative In= 6,289 Turn 4: Input= 3,233 tokens | Output= 317 tokens | Messages= 7 | Cumulative In= 9,522 Turn 5: Input= 3,803 tokens | Output= 30 tokens | Messages= 9 | Cumulative In= 13,325 Turn 6: Input= 4,033 tokens | Output= 392 tokens | Messages=11 | Cumulative In= 17,358 Turn 7: Input= 5,026 tokens | Output= 321 tokens | Messages=13 | Cumulative In= 22,384 ============================================================ 🔄 Compaction occurred! 15 messages condensed into one summary block Summary after compaction: ## Progress summary **Task**: Process 5 support tickets one at a time. For each ticket: get_next_ticket → classify_ticket → search_knowledge_base → set_priority → route_to_team → draft_response → mark_complete. Keep going until the queue is empty (the 5 tickets are expected in total). **Completed so far (2 of 5):** - **TICKET-1** (John Williams, tracking number 1Z856942470332 reported invalid): classified shipping, priority medium, routed to logistics-team. The draft response made no unsupported claims and said the logistics team would verify the number. Marked complete. - **TICKET-2** (Alex Jones, order #ORD-66101: received Yellow Widget Lite instead of Green Device Max; the subject line said "change delivery address", which doesn't match the description): classified shipping, priority medium, routed to logistics-team. The draft apologized, said logistics would arrange the correct item and return instructions, and offered to pass on a new address if needed. Marked complete. **Notes on the tools:** - The KB only has the categories billing, technical, and account. "shipping" returns an error. For shipping tickets, I searched the nearest available category (billing for return/refund topics, technical for tracking). Neither had shipping-specific content, so the drafts avoided inventing policy details. - KB content seen: the billing refund policy (refunds take 5-7 business days for valid cancellations, pro-rated for annual plans), payment methods (Visa, Mastercard, Amex, PayPal; update them in Account Settings), and the billing cycle. Technical covers common errors, system requirements (4GB RAM, Chrome 90+/Firefox 88+/Safari 14+), and troubleshooting (clear cache, incognito, enable JavaScript). - Valid teams: billing-team, tech-support, account-services, product-success, logistics-team. - Independent calls (classify, search, priority, route) can be made in parallel. Draft and mark_complete come after them. **Next step**: Call get_next_ticket for TICKET-3, then repeat the full workflow for TICKET-3, 4, and 5. Stop when the queue returns an empty error. Then give a brief final summary of all processed tickets. ============================================================ Turn 8: Input= 5,661 tokens | Output= 758 tokens | Messages=15 | Cumulative In= 28,045 Turn 9: Input= 2,087 tokens | Output= 30 tokens | Messages= 1 | Cumulative In= 30,132 Turn 10: Input= 2,303 tokens | Output= 355 tokens | Messages= 3 | Cumulative In= 32,435 Turn 11: Input= 3,145 tokens | Output= 355 tokens | Messages= 5 | Cumulative In= 35,580 Turn 12: Input= 3,751 tokens | Output= 30 tokens | Messages= 7 | Cumulative In= 39,331 Turn 13: Input= 3,982 tokens | Output= 314 tokens | Messages= 9 | Cumulative In= 43,313 Turn 14: Input= 4,890 tokens | Output= 426 tokens | Messages=11 | Cumulative In= 48,203 Turn 15: Input= 5,569 tokens | Output= 30 tokens | Messages=13 | Cumulative In= 53,772 ============================================================ 🔄 Compaction occurred! 15 messages condensed into one summary block Summary after compaction: ## Progress summary **Task**: Process 5 support tickets one at a time. For each ticket: get_next_ticket → classify_ticket → search_knowledge_base → set_priority → route_to_team → draft_response → mark_complete. Keep going until the queue is empty (the 5 tickets are expected in total). **Completed so far (4 of 5):** - **TICKET-1** (John Williams, tracking number reported invalid): shipping, medium, logistics-team. Draft made no unsupported claims. Marked complete. - **TICKET-2** (Alex Jones, wrong item received on order #ORD-66101; the subject line didn't match the description): shipping, medium, logistics-team. Marked complete. - **TICKET-3** (Alex Jones, password reset emails not arriving): account, medium, account-services. The draft cited the KB (emails come from noreply@support.example.com, the link expires in 1 hour). It flagged that the ticket email (customer8695@) differs from the email in the description (customer6750@) and asked the customer to confirm which is registered. Marked complete. - **TICKET-4** (John Smith, unrecognized $49.99 charge on 2026-10-20): billing, high, billing-team. The draft explained the billing cycle, did not claim to know what the charge was, mentioned refund policy conditionally, and asked for the last four digits only. Marked complete. **Currently in progress: TICKET-5** (just fetched, no actions taken yet): - Jane Brown, customer4919@example.com, subject "Account locked after failed login". She says the account was locked after too many wrong password attempts. She now remembers her password, but the unlock email link isn't working. Pre-set category: account. Created 2026-10-19. - Plan: classify as account, search the account KB (e.g. "account locked unlock"), set priority (medium, or high since she is locked out), route to account-services. Then draft a response that doesn't invent unlock policy beyond what the KB returns, and mark complete. The KB has password_reset (emails from noreply@support.example.com, check spam, link expires in 1 hour), 2fa_backup, and email_change, so there may be no lockout-specific entry. If so, say the Account Services team will verify her identity and help unlock the account, and suggest requesting a new link since the old one may have expired. **Notes on the tools:** - The KB only has the categories billing, technical, and account. "shipping" returns an error. - Valid teams: billing-team, tech-support, account-services, product-success, logistics-team. - Independent calls (classify, search, priority, route) can be made in parallel. Draft and mark_complete come after them (these two can also go in parallel). **Next steps**: Finish TICKET-5 with the full workflow. Then call get_next_ticket once more to confirm the queue is empty. Then give a brief final summary of all 5 processed tickets, noting the oddities flagged: the TICKET-2 subject/description mismatch, the TICKET-3 email mismatch, and the absence of a shipping KB category. ============================================================ Turn 16: Input= 5,840 tokens | Output=1,027 tokens | Messages=15 | Cumulative In= 59,612 Turn 17: Input= 2,356 tokens | Output= 309 tokens | Messages= 1 | Cumulative In= 61,968 Turn 18: Input= 3,217 tokens | Output= 386 tokens | Messages= 3 | Cumulative In= 65,185 Turn 19: Input= 3,910 tokens | Output= 460 tokens | Messages= 5 | Cumulative In= 69,095 ============================================================ OPTIMIZED RESULTS (WITH COMPACTION) ============================================================ Total turns: 19 Compactions: 2 Input tokens: 69,095 Output tokens: 5,988 Total tokens: 75,083 ============================================================
With context compaction enabled, token usage per turn no longer grows linearly: it drops back after each compaction event, and the following turns start from the summary instead of the full history.
Compared to the baseline, the run uses far fewer input tokens (the comparison table below has the totals). We've also printed the summary returned by each compaction, showing how prior ticket details were condensed.
Let's look at the final response after processing all 5 tickets with compaction enabled.
All 5 tickets are processed and the queue is empty. | Ticket | Customer | Issue | Category | Priority | Team | |---|---|---|---|---|---| | TICKET-1 | John Williams | Tracking number reported invalid | shipping | medium | logistics-team | | TICKET-2 | Alex Jones | Wrong item received on order #ORD-66101 | shipping | medium | logistics-team | | TICKET-3 | Alex Jones | Password reset emails not arriving | account | medium | account-services | | TICKET-4 | John Smith | Unrecognized $49.99 charge | billing | high | billing-team | | TICKET-5 | Jane Brown | Account locked, unlock link not working | account | high | account-services | Each ticket has a draft response and is marked complete. Three things to check: - **TICKET-2:** The subject line doesn't match the description. I went by the description. - **TICKET-3:** The email on the ticket (customer8695@) differs from the one in the description (customer6750@). The draft asks the customer to confirm which one is registered. - **Knowledge base gaps:** The knowledge base has no shipping category, so the shipping drafts make no policy claims. It also has no entry on account lockouts. The TICKET-5 draft only cites the password-reset link expiry and hands the unlock to Account Services.
Comparing Results
With compaction enabled, we can see a clear difference between the two runs in token savings, while preserving the quality of the workflow and final summary.
Here's what changed with automatic context compaction:
-
Context resets after several tickets - When a few tickets' worth of tool results pushes a turn past 5k input tokens, the loop calls
compact_before_next_turn()and the runner:- Finishes the current turn's tool calls, then sends a compaction request
- Receives a server-generated summary as a
compactionblock - Replaces the conversation history (detailed classifications, KB searches, responses) with that block
- Continues with only the summary
-
Input tokens stay bounded - Instead of accumulating to 100k+ as we process more tickets, input tokens reset after each compaction. When processing Ticket #5, we're NOT carrying the full tool results from Tickets #1-4.
-
Task completes successfully - The workflow continues smoothly through all tickets without hitting context limits
-
Quality is preserved - The summaries retain critical information:
- Tickets processed with their IDs
- Categories and priorities assigned
- Teams routed to
- Overall progress status
All tickets are still properly classified, prioritized, routed, and responded to.
-
Natural workflow - This mirrors how real support agents work: resolve a ticket, document it briefly in the system, close it, move to the next one. You don't keep every knowledge base article and full response draft open while working on new tickets.
Let's visualize the token savings:
====================================================================== TOKEN USAGE COMPARISON ====================================================================== Metric Baseline With Compaction ---------------------------------------------------------------------- Input tokens: 109,269 69,095 Output tokens: 4,238 5,988 Total tokens: 113,507 75,083 Compactions: N/A 2 ====================================================================== 💰 Token Savings: 38,424 tokens (33.9% reduction)
How Compaction Works Under the Hood
When you call runner.compact_before_next_turn(), the tool runner:
- Lets the current turn finish - pending tool calls are run and their results appended first, because the API can't compact a conversation that ends with an unanswered tool call
- Sends a compaction request - the same model, system prompt, tools and messages, plus
compaction={"type": "summarize"}. Nothing is sampled after the summary, so this request returns only the summary - Receives a
compactionblock - the API summarizes the conversation server-side (completed tickets, progress, what to do next) and returns it as a single content block withstop_reason == "compaction" - Replaces history - the runner swaps the entire message list (including all tool results) for that one block, which must be sent back first and unmodified
- Resumes processing - the next request continues from the compressed context, and Claude picks up the next ticket
Customizing Compaction Configuration
You can customize how compaction works to fit your specific use case. Here are the key knobs:
Adjusting the Threshold
With runner-driven compaction the threshold is your own loop logic, so it can be any value or any condition (for example, "after every third mark_complete call"):
The threshold should not be set too low, otherwise the summary itself could trigger a compaction. We set a threshold of 5,000 tokens for demonstration purposes, but in practice, experiment with different settings to find what works best for your workflow.
Here some general guidelines:
-
Low thresholds (5k-20k):
- Use for iterative task processing with clear boundaries
- More frequent compaction, minimal context accumulation
- Best for sequential entity processing
-
Medium thresholds (50k-100k):
- Multi-phase workflows with fewer, larger natural checkpoints
- Balance between context retention and management
- Suitable for workflows with expensive tool calls
-
High thresholds (100k-150k):
- Tasks requiring substantial historical context
- Less frequent compaction preserves more raw details
- Higher per-call costs but fewer compactions
If all you need is a fixed token trigger of 50k or more, you don't need loop logic at all: pass a context_management compaction edit and the API compacts automatically (see the compaction docs(opens in new tab)). The runner refuses to combine that edit with compact_before_next_turn(), so pick one approach per runner.
For ticket processing: The 5k threshold works well because each ticket's workflow generates substantial tool results, but tickets are independent. After resolving Ticket A, you don't need its detailed KB searches when processing Ticket B.
Custom Summary Instructions
You can pass instructions to replace the server's default summarization prompt. This is especially useful for customer service workflows where you need to preserve specific types of information.
For example, we could define a custom prompt based on our requirements:
- Ticket summaries for all completed tickets
- Categories and priorities assigned
- Teams routed to
- Progress status (tickets completed, tickets remaining)
- Next steps in the workflow
Compaction Without Tools: Simple Chat Loop
While the examples above focus on tool-heavy agentic workflows, context compaction is also valuable for simple conversational applications where users drive the conversation.
Note: compact_before_next_turn() is a tool-runner method. In a plain chat loop you can either send compaction={"type": "summarize"} on a client.beta.messages.create() call yourself and put the returned block first in messages, or implement the summarization manually as shown below, which works with any model and gives you full control over the summary prompt.
Consider a chat application where users are having extended conversations with Claude—discussing complex topics, iterating on ideas, or working through problems. As the conversation grows, you face the same context accumulation challenges.
The Difference: Instead of tool use triggering token growth, it's the back-and-forth conversation itself. Each exchange adds messages to the history:
- User asks a question
- Claude provides a detailed response
- User asks for clarification or elaboration
- Claude responds with more context
- This repeats dozens or hundreds of times
Without compaction, by turn 50 you're sending the entire conversation history (all 50 exchanges) on every API call.
The Solution: Implement compaction manually in your chat loop using the same pattern:
- Track token usage after each turn
- When threshold is exceeded, request a summary
- Replace conversation history with the summary
- Continue the conversation with compressed context
Let's see how to implement this:
Chat with Claude (type 'quit' to exit, or just hit Enter to continue)
This is a demonstration - try having a conversation and watch compaction trigger
============================================================
You: Help me understand how Python decorators work
Claude: # Python Decorators
A decorator is a function that takes another function and returns a modified or enhanced version of it, without you changing the original function's code.
## 1. The building blocks
Decorators rely on two Python features.
**Functions are objects.** You can assign them to varia ...
[Tokens: 20 in + 1497 out = 1517 total]
You: Can you show me an example with a timing decorator?
Claude: # Timing Decorator: Full Example
## Basic version
```python
import time
import functools
def timer(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
elapsed = time.perf_counter() - start
p ...
[Tokens: 1536 in + 1437 out = 2973 total]
You: How would I make a decorator that takes arguments?
Claude: # Decorators That Take Arguments
## The core idea
A normal decorator is a function that takes a function. To pass arguments, add one more layer: a function that takes the arguments and **returns** a decorator.
```
decorator_factory(args) → decorator(func) → wrapper(*args, **kwargs)
```
## Th ...
[Tokens: 2992 in + 1641 out = 4633 total]
============================================================
🔄 Compacting conversation... 6 messages →
1 message
============================================================
Final conversation messages: <summary>
## Continuation Summary
### 1. Task Overview
The user is learning Python decorators through a conversational Q&A. There is no fixed deliverable or file output. The goal is clear, accurate, example-driven explanations. No explicit constraints were given beyond the questions asked. The task is not "unfinished" in a strict sense; the conversation is open-ended and the user may ask further follow-ups.
### 2. Current State
Three user questions have been answered, all in chat (no files created or modified):
1. **"Help me understand how Python decorators work"** – Covered: functions as first-class objects, nested/returned functions, manual decoration (`greet = shout(greet)`), `@` syntax as sugar, `*args/**kwargs` wrappers, `functools.wraps`, decorators with arguments (brief), a timing example, stacking order (`@a @b def f` = `a(b(f))`), a table of common built-in decorators (`staticmethod`, `classmethod`, `property`, `lru_cache`, `dataclass`), and a mental model. Ended by offering to go deeper on class-based decorators, decorating classes, and closures.
2. **"Can you show me an example with a timing decorator?"** – Provided: a basic `timer` using `time.perf_counter()`, usage with args/return values, a `try/finally` version so timing prints even on exceptions, a configurable version `timer(label=None, log=False)` (requires parentheses), an `async_timer` for coroutines, a step-by-step call flow, and a note pointing to `timeit` for serious benchmarking.
3. **"How would I make a decorator that takes arguments?"** – Provided: the three-layer pattern (factory → decorator → wrapper), the desugaring `@repeat(3)` = `repeat(3)(func)`, closures as the reason inner layers see outer args, a `require_role` access-control example, an optional-parentheses pattern (`func=None, *, times=2`), common mistakes (missing parentheses, missing returns, omitting `wraps`), and a class-based alternative (`__init__` stores args, `__call__` takes the function).
### 3. Important Discoveries / Decisions
- Consistent teaching approach: build from first principles, show the desugared equivalent, then give practical examples.
- Always include `@functools.wraps(func)` in examples, and explain why.
- Use `time.perf_counter()` over `time.time()` for timing.
- Use `try/finally` in timing decorators so exceptions don't skip the measurement.
- Async functions need an `async def wrapper` with `await`; a sync wrapper would only time coroutine creation.
- No errors or failed approaches have occurred.
### 4. Next Steps
- No required actions remain. Await the user's next question.
- Likely follow-ups, already offered or naturally adjacent: class-based decorators (a class with `__call__` that wraps a function, and keeping state such as call counters), decorating classes, closures in depth, `lru_cache`/memoization, decorators on methods (handling `self`), stacking order subtleties, a retry/caching/validation decorator example, type hints for decorators (`ParamSpec`, `TypeVar`), and testing decorated functions (`__wrapped__`).
### 5. Context to Preserve
- Style: Markdown with headers, short code blocks with printed output shown in comments or output blocks, brief explanatory prose, and a concise summary or mental model at the end. Code is runnable and minimal.
- The user appears to be a learner, so build up concepts incrementally and avoid unexplained jargon.
- Promises made: offered to go deeper on class-based decorators, decorating classes, and how closures work, if the user wants.
</summary>
Demo complete! In a real application, this loop would continue with user input.Understanding the Chat Loop Pattern
The example above demonstrates manual compaction in a conversational context. Here's how it works:
Key Components:
- Token Tracking: After each response, calculate total tokens (input + output + cache tokens)
- Threshold Check: When total exceeds threshold, trigger compaction
- Summary Request: Send the same structured SUMMARY_PROMPT to Claude
- History Replacement: Replace entire message history with just the summary
- Continue: Next user message builds on the summary, not full history
When to Use This Pattern:
- Extended brainstorming sessions: Users exploring ideas with Claude over many turns
- Learning conversations: Tutorials or explanations that span dozens of exchanges
- Iterative refinement: Users providing feedback on drafts, designs, or solutions
- Chat applications: Any multi-turn conversation interface
Key Differences from Tool Runner:
| Aspect | Tool Runner (compact_before_next_turn) | Chat Loop (Manual) |
|---|---|---|
| Trigger | Your threshold check schedules it; runner picks the moment | You implement threshold check |
| Summary | Generated server-side, returned as a compaction block | You make explicit API call with your own prompt |
| History Management | Runner replaces messages | You manually replace list |
| Use Case | Agentic workflows with tools | User-driven conversations |
Production Considerations:
- Adjust threshold: Use larger thresholds for real applications
- Customize summary prompt: Tailor to your conversation type (brainstorming vs. technical support vs. tutoring)
- Show user indicators: Display a message like "Summarizing conversation..." so users understand the pause
- Preserve key context: Ensure the summary prompt captures domain-specific information your users care about
This pattern gives you full control over when and how compaction happens, making it ideal for conversational applications where you aren't using the tool runner.
Limitations and Considerations
While automatic context compaction is powerful, there are important limitations to understand:
Measuring the Threshold
Watch what you count: this cookbook triggers on usage.input_tokens. With prompt caching enabled most of the history shows up under cache_read_input_tokens instead, and server-side tool loops (web search, code execution) report several usage.iterations per turn. Sum the counters that reflect real history size in your workflow, or the trigger fires too late or too early.
Timing: a scheduled compaction runs only after the current turn's tool calls have been answered, and it is skipped (with a warning) if the run ends on a turn with unanswered tool calls.
Information Loss
Trade-off: Summaries inherently lose some information. While Claude is good at identifying key points, some details will be compressed or omitted.
In ticket processing:
- ✅ Retained: Ticket IDs, categories, priorities, teams, outcomes, progress status
- ❌ Lost: Full knowledge base article text, complete drafted response text, detailed classification reasoning
This is usually acceptable, you don't need every KB article and full response text in perpetuity, just the completion records.
Mitigation:
- Use custom summary prompts to preserve critical information
- Set higher thresholds for tasks requiring extensive historical context
- Structure your tasks to be modular (each phase builds on summaries, not raw details)
When NOT to Use Compaction
Avoid compaction for:
- Short tasks: If your task completes within 50k-100k tokens, compaction adds unnecessary overhead
- Tasks requiring full audit trails: Some tasks need access to ALL previous details
- Highly iterative refinement: Tasks where each step critically depends on exact details from all previous steps
When TO Use Compaction
Compaction is ideal for:
- Sequential processing: Like our ticket workflow—process multiple items one after another
- Multi-phase workflows: Where each phase can summarize progress before moving on
- Iterative data processing: Processing large datasets in chunks or entities one at a time
- Extended analysis sessions: Analyzing data across many entities
- Batch operations: Processing hundreds of items where each is independent
Ticket processing is a perfect use case because:
- Each ticket workflow is largely independent
- You need completion summaries, not full tool results
- Natural compaction points exist (after completing several tickets)
- The workflow is iterative and sequential
Summary
Automatic context compaction is a powerful feature that enables long-running agentic workflows to exceed typical context limits. In this cookbook, we've explored compaction through a customer service ticket processing workflow.
Next Steps
Try implementing compaction in your own workflows:
- Identify natural compaction points (after processing each item, completing each phase, etc.)
- Start with an aggressive threshold (5k-10k) if you have clear per-item boundaries
- Use custom summary prompts to preserve critical information
- Monitor when compaction triggers and verify quality is maintained
- Adjust threshold based on your specific needs
For more on effective context management, see Effective Context Engineering for AI Agents(opens in new tab).