A conversation still carrying last week's context
A conversation that stays open for a week can feel economical. The assistant already knows the project, the earlier decisions, and the failed approaches. Starting again seems wasteful. Yet continuity has a cost: depending on the client and protocol, much of that history may become input again on the next request.
The useful comparison is not between a good memory and no memory. It is between carrying the full transcript and carrying a deliberately maintained working state.
§ The input sent on each turn
One continuous conversation: In a typical chat workflow, each request includes earlier messages alongside the new question. Previous answers, pasted documents, and tool results can therefore contribute to input usage repeatedly. A short follow-up does not necessarily make a small request.
Compact working sessions: Each new session starts with a brief containing the objective, constraints, decisions, and necessary source material. The request can be smaller, but only if the client actually omits the old transcript. Opening a new tab is not proof that less context is being sent.
§ The bill behind a small follow-up
One continuous conversation: Consider an illustrative sequence of 20 requests whose input grows from 2,000 tokens to 40,000 tokens in 2,000-token steps. Total input is 420,000 tokens, not 40,000. This counts input only; generated output is additional. The arithmetic describes repeated processing, not a particular provider's price.
Compact working sessions: If each of those 20 requests instead fits within a 2,000-token working context, total input is 40,000 tokens. That is a useful budget comparison, not a promise of equivalent results. Preparing summaries takes work and may consume tokens, and some tasks genuinely require more evidence than the smaller budget allows.
§ Cached history and removed history
One continuous conversation: Prompt caching can reduce the cost of eligible repeated input and may improve response time. But a long transcript is not automatically a cache hit. Eligibility, prefix matching, retention, and pricing depend on the model and service. Cached tokens also remain part of the context the model receives.
Compact working sessions: Removing irrelevant history reduces the input itself. That is different from receiving a discount on repeated input. A stable brief can also benefit from caching where supported. Check actual usage and billing details rather than assuming either strategy always costs less.
§ Continuity and answer quality
One continuous conversation: The transcript preserves nuance: rejected options, qualifications, and the reasons behind decisions. That can be valuable during investigation. It can also preserve obsolete requirements and contradictory instructions. A large context window permits more material; it does not guarantee equal attention to every detail.
Compact working sessions: A curated brief makes current instructions easier to identify. Its weakness is editorial loss. A summary may erase an exception or turn a tentative idea into an apparent decision. Preserve exact requirements and identify unresolved questions explicitly. Keep original evidence accessible when wording matters.
§ Latency and tool-heavy work
One continuous conversation: Repository searches, command output, and repeated file reads can expand history quickly. Larger inputs may increase processing time, although caching, model choice, routing, and output length also affect latency. A slower response is not sufficient evidence that context length is the cause.
Compact working sessions: Before a new task, retain the relevant files, current findings, and next action rather than every intermediate log. In tools such as Claude Code, inspect the client's context-management behavior. Automatic compaction can help, but it is not the same as a verified, task-specific handoff.
§ A small comparison worth running
On Vana API, the OpenAI-compatible endpoint at https://api.vanaapi.com also supports Anthropic Messages and Gemini native protocols. Billing is per real token, with no monthly fee. This makes input discipline worth measuring, while the client still determines what history it sends.
Create an account, then compare two representative tasks: one with the existing transcript and one with a checked working brief. Keep the model and output requirements constant. Record input usage, cache usage where reported, total cost, latency, and corrections needed. Keep long conversations where their evidence earns its place; shorten them where they merely carry yesterday's work.