Guide · Updated 2026-11-08

Context compression: the complete guide for LLMs and AI agents

Context compression is the practice of shrinking what you send to a large language model (tool output, logs, files, chat history, retrieved documents) so the model sees the information it needs in fewer tokens. It can reduce input cost, keep requests within context limits and remove distracting material. The tradeoff is information loss: evaluate whether the compressed context still supports the answer, and include compression overhead when measuring latency. This guide covers the methods, how query-aware compression differs, how to measure it and which tools do what.

What is context compression?

Context compression reduces the number of tokens in an LLM's input while keeping the information that matters for the task. It is sometimes called prompt compression, token compression, token reduction or context pruning. The input can be anything that lands in the context window: a system prompt, chat history, a 5,000 line log, a diff, an MCP tool result, a web page or a batch of RAG chunks.

Why context compression matters

Cost: providers bill per input token, and in agent loops input usually dwarfs output because every turn resends history and tool results.
Context window limits: big tool results push sessions into compaction, truncation or errors such as MCP output too large. See LLM context window too long and Claude Code MCP output too large.
Quality: long, noisy contexts dilute attention, a failure mode often called context rot.
Latency: fewer uncached input tokens can reduce prefill work, but a compression call adds processing and network time. Measure total task latency; a smaller prompt alone does not prove a faster response.

Context compression methods

1. Truncation

Cut the input at a length limit. Cheap and predictable, but it drops whatever falls past the cut, which is often the error at the end of a log. See compression vs truncation.

2. Summarization (abstractive)

Ask a model to rewrite the context shorter. Good for narrative history, risky for code, IDs, numbers and error strings because rewritten text can drift from the source. See compression vs summarization and extractive vs abstractive.

3. Token pruning

A small model scores tokens and deletes low-information ones (LLMLingua, LLMLingua-2, Selective Context, hosted APIs such as Bear). High ratios are possible, but tokens disappear inside lines, which can break code and identifiers. See how prompt compression works.

4. Extractive, line-level selection

Keep or drop whole original lines or spans. What is kept is verbatim, so paths, stack traces and code stay exact. See verbatim lines vs token pruning.

5. Rule-based filters

Command-specific rules strip known noise from shell output (rtk is the best-known example). Fast and local, but blind to what you are asking.

6. Retrieval instead of compression

Index the codebase or corpus and fetch only relevant chunks or symbols (claude-context, jCodeMunch). This sends less in the first place but does not shrink a big result once it has been produced.

7. Soft or latent compression

Soft prompt methods such as gist tokens and AutoCompressors encode context into learned representations rather than ordinary text. KV cache compression instead changes the attention state retained during inference. These are different mechanisms, generally requiring model or serving integration; they are not interchangeable with a text compressor placed before an arbitrary hosted API.

8. Session compaction

Coding agents summarize or clear history when the window fills (/compact, /clear). Useful, but lossy. See /compact vs /clear vs compression.

Query-aware context compression

Most methods decide what to keep from the content alone. Query-aware compression also looks at the question being asked right now, so the same 2,000 line log keeps different lines depending on whether you asked why the build failed or which version was installed. LongLLMLingua and some hosted APIs are question-aware at the token level. SuperCompress is query-aware at the line level: the hosted Neural Keep model keeps the original lines that answer the current question, verbatim, and drops the rest.

Worked example: compress a build log against a question

Suppose the question is “Why did deployment fail, and which revision was affected?” The following synthetic log illustrates selection; it is not output from a SuperCompress benchmark.

10:00:00 INFO revision=8c42e71 environment=production
10:00:01 INFO downloaded dependencies
10:00:02 INFO cache restored
10:00:03 INFO build completed
10:00:04 ERROR migration 042 failed: permission denied for schema public
10:00:04 ERROR deployment aborted; previous revision remains active

A useful extract keeps the revision, root error and deployment outcome:

10:00:00 INFO revision=8c42e71 environment=production
10:00:04 ERROR migration 042 failed: permission denied for schema public
10:00:04 ERROR deployment aborted; previous revision remains active

This removes three of six lines, but that is a line count, not a measured token reduction. A summary saying only “deployment failed” would lose the permission error and revision. Selecting only the error line would lose the affected revision and the fact that the previous deployment remains active. Verbatim extraction preserves selected text, but it can still omit necessary evidence. If the question changes to “Was the cache restored?”, the selected lines should change too.

How LangChain, Google ADK and Agno implement context compression

Frameworks apply compression at different boundaries. Choose the boundary that produces your excess context before choosing a compressor.

LangChain Deep Agents: its autonomous context compression tool lets the agent trigger compaction at a useful task boundary. This condenses older working history; it serves a different role from selecting evidence inside a single tool response. Availability and defaults depend on the CLI or SDK version.
Google ADK: context compaction summarizes older session events while retaining recent interactions. Python exposes EventsCompactionConfig, including token-based settings and turn-based compaction_interval and overlap_size. Use the configuration for your installed SDK version; language implementations differ.
Agno: tool-result compression uses compress_tool_results=True or a CompressionManager to summarize individual tool results after a threshold. It does not shorten system or user messages, and the documentation notes that summaries can lose information or exceed the original length.

These approaches can coexist with retrieval or query-aware extraction. Avoid repeatedly summarizing already compressed material without evaluation: each transformation can remove evidence that a later turn needs.

Context compression for coding agents

Coding agents (Claude Code, Cursor, Codex, Windsurf, Cline, Gemini CLI, OpenCode and others) spend most tokens on tool results: file reads, grep, test output, build logs and MCP responses. Agent context pruning works best at that boundary, before the result lands in history and is resent every turn. Guides: reduce Claude Code token usage, Cursor, Codex, MCP tool output compression, approaches for coding agents.

Context compression for RAG

Retrieved chunks are usually only partly relevant. Compressing them against the user question before generation cuts tokens and can reduce distraction. Keep source lines verbatim when answers must be quotable or cited.

How to measure context compression

Tokens in vs tokens kept on your real traffic (the SuperCompress API returns original_tokens and kept_tokens on every call).
Task success with and without compression on the same tasks.
Cost per finished task on your provider bill, since caching, retries and extra turns change the picture. See does context compression lower your bill?

A practical evaluation protocol

Fix a set of representative tasks and their expected evidence: error strings, source citations, identifiers, constraints and decisions. Include questions that require multiple distant passages.
Run an uncompressed baseline and compressed variant using the same downstream model, settings and tools. Count tokens using the target model's tokenizer; record compressor version and settings.
Check answer correctness and evidence retention, including negations, units and source attribution. A high compression ratio is not a quality score.
Record compression time, model time, retries, extra turns and billed input/output tokens. Separate cached from uncached input. Compare total cost per successful task and median and tail latency.
Inspect failures and set a fallback. For oversized output, errors or uncertain evidence retention, preserve the original when it fits, or retrieve a narrower source span and reevaluate.

For one request, net savings are baseline generation cost − (compression cost + compressed generation cost). For an agent session, also include repeated context, cache effects and recovery turns. Do not infer a bill reduction from a token-reduction percentage alone.

Our published methodology and limitations are on the benchmarks page.

Context compression tools compared

Tool	Best if	What it optimizes	Runs	Licence
SuperCompress	your agent keeps pulling big tool results, logs or retrieved docs and only a few lines matter for the question at hand, and you want those lines kept verbatim.	Input context (tool output, logs, files, MCP results, RAG)	Hosted API, plus an MCP tool and hooks for coding agents	Hosted service
Headroom	you want a broad local compression layer with a library, proxy, agent wrapper or MCP server.	Input context and history	Locally (library, proxy, MCP, agent wrap)	Apache-2.0
LLMLingua	you run your own Python pipeline, can host a small model next to it, and want a well-cited research method.	Prompts and retrieved context	Locally, in your Python pipeline	MIT
Compresr	you want a hosted compression API for RAG and Q&A pipelines and are fine with token-level output.	Prompts, documents, RAG context	Hosted API (on-premises option)	Closed hosted service
Bear (The Token Company)	you want a one-line wrapper around an OpenAI or Anthropic client that deletes low-signal tokens.	Prompts, chat history, search results	Hosted API (enterprise VPC option)	Closed hosted service
rtk	your pain is noisy shell command output and you want a free, local, rule-based filter.	Shell command output	Locally (CLI proxy and hook)	Open source
jCodeMunch	most of your waste is the agent reading whole files to find one function.	Source code reads	Locally, as an MCP server	Custom dual-use licence
claude-context	retrieval quality is the goal and you already run, or are happy to run, vector infrastructure.	What code gets read	MCP server backed by Milvus or Zilliz Cloud	Open source (needs vector DB and embeddings)
Caveman	you want a local feature bundle spanning input compression and terse replies; the standalone skill changes reply style.	Input context and model replies in v4	Local CLI/proxy and optional skill	Apache-2.0

Full write-ups: all comparisons, Headroom alternatives, LLMLingua alternatives, Compresr alternatives, tools compared.

Try query-aware compression

Coding agents: one command installs an MCP tool for each detected agent, plus hooks where supported.

npm install -g supercompress-proxy && supercompress setup

Apps: call POST /api/v1/compress with your context and question. See the API reference.

Primary sources and further reading

The framework descriptions above are based on their documentation, reviewed November 8, 2026. Framework features change; check your installed version.

LangChain: Autonomous context compression — agent-triggered history compaction.
Google ADK: Context compression — session event compaction and configuration.
Agno: Context Compression — tool-result summarization, thresholds and limitations.
Microsoft Research: LLMLingua series — prompt compression research and implementations.

FAQ

What is context compression?

Context compression shrinks the input sent to an LLM, such as tool output, logs, files, chat history or retrieved documents, so the model gets the information it needs in fewer tokens.

What is the difference between context compression and prompt compression?

They overlap. Prompt compression usually refers to shrinking a prompt, often by token pruning. Context compression is broader and covers anything in the context window, including tool results and history in agent loops.

What is query-aware context compression?

Compression that uses the current question to decide what to keep, so the same input keeps different content depending on what is being asked.

Does context compression hurt accuracy?

It can if it drops what the answer needs or rewrites facts. Extractive, query-aware methods that keep original lines verbatim reduce that risk. Always test task success with and without compression.

How do I reduce Claude Code, Cursor or Codex token usage?

Check what fills the window, keep instruction files short, disable unused MCP servers, start fresh sessions per task, and compress bulky tool output against the current question, for example with npm install -g supercompress-proxy && supercompress setup.

How do I fix MCP output too large errors?

Compress or filter the tool result before it reaches the model, or ask the tool for a narrower result. Query-aware compression keeps the lines relevant to the question.

Is summarization a form of context compression?

Yes, abstractive compression. It is useful for narrative history but can alter code, numbers and identifiers, so extractive methods are safer for technical content.

Get API key Coding agents guide All comparisons