Working Memory: Context Windows, Scratchpads, and State
Working Memory: Context Windows, Scratchpads, and State
Context windows are not memory
LLMs have a context window, not memory. It’s temporary and expensive. Working memory is how you manage that window: what to include, what to omit, and what to summarize.
Scratchpads: internal state that keeps the agent consistent
A scratchpad is a structured internal note like:
- Current goal
- Known constraints
- What has been done
- What’s next
This reduces drift because the agent doesn’t “forget” the plan halfway.
Practical token budgeting
In production, you must treat tokens like money:
- Summarize tool outputs
- Keep only the last N turns verbatim
- Promote stable facts to long-term memory
State machines beat free-form memory
For workflows (onboarding, ticketing, checkout), a simple state machine is often better than dumping conversation context into the LLM. Use the LLM for reasoning, not for tracking state.
