LLMLingua alternative · Updated 2026-08-03
SuperCompress vs LLMLingua / LLMLingua-2
LLMLingua and LLMLingua-2 are Microsoft research compressors that prune tokens with a small LM / classifier. SuperCompress is a query-aware evidence selector: it keeps original lines that answer the current question, runs ~60ms on CPU, and ships a hosted API + MCP for coding agents.
How we compare (methodology)
Do not rank tools only by “% tokens removed.” Aggressive deletion can destroy the answer. Compare on four axes:
- Mechanism — SuperCompress: query-aware block/line keep (extractive). LLMLingua: iterative budget-controlled token prune with a small LM. LLMLingua-2: single-pass token classification (e.g. XLM-RoBERTa-class SLM) for faster research pruning.
- Latency & hardware — SuperCompress: tiny CPU policy (~60ms), no GPU required. LLMLingua-1: often heavier / GPU-friendly. LLMLingua-2: faster than v1 but still a local SLM dependency.
- Answer containment — SuperCompress publishes held-out gold-answer keep (≥98% on primary suites) with ~65% token-weighted cut. For LLMLingua papers, cite their reported metrics on their datasets; re-run both on your RAG/agent dumps before claiming a winner.
- Product path — SuperCompress:
pip install, hosted/api/v1/compress, MCP for Cursor/Claude Code/Codex. LLMLingua: research library / DIY integration; no equivalent MCP agent product.
Side-by-side
| Factor | LLMLingua | LLMLingua-2 | SuperCompress |
|---|---|---|---|
| Core idea | Budgeted token prune via small LM | Single-pass token classifier (SLM) | Query-aware evidence selection |
| Keeps original lines? | Partial (token delete can fragment) | Partial (token delete) | Yes — extractive keep |
| Typical runtime | Heavier / often GPU | Faster than v1; local SLM | ~60ms CPU |
| Query-aware? | Budget / instruction aware | Task-aware classification | Scores against current question |
| Hosted API / MCP agents | DIY | DIY | Yes |
| Published answer-keep gates | Paper datasets | Paper datasets | Held-out ≥98% answer keep |
| Best default for | Research / self-host experiments | Faster research pruning | Production APIs & coding agents |
Fair benchmark recipe
- Pick a fixed held-out set of (context, question, gold answer) — agent dumps, RAG chunks, or your own logs.
- Run SuperCompress and LLMLingua-2 at matched keep ratios (e.g. keep ~35% tokens).
- Score gold-answer containment in the compressed text (exact/soft match), not only ROUGE against a paraphrase.
- Record wall-clock latency and peak memory (CPU vs GPU).
- Optionally ask a judge LLM on original vs compressed — but treat containment as the hard gate.
Our public numbers live on /benchmarks. Replicate with your data before switching tools.
When LLMLingua-2 still makes sense
- You already run Microsoft’s stack and want paper-faithful token pruning.
- You need extreme token deletion on prose and accept local SLM ops.
- You’re reproducing academic baselines — cite LLMLingua; ship SuperCompress for production.
KV cache note
Neither SuperCompress nor typical LLMLingua usage “compresses inside KV.” Both shrink the prompt text before the model call. SuperCompress’s optional cache_prefix only wraps text for provider prompt/prefix caching — it still does not edit KV.
Get started
pip install supercompress
# or
curl -X POST https://www.supercompress.dev/api/v1/compress \
-H "Authorization: Bearer $SUPERCOMPRESS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"context":"...","query":"..."}'Coding agents: docs.supercompress.dev/coding-agents