Context compression for LLM applications that runs locally — 50–90% fewer tokens for the same answers, with nothing shipped to a third-party service to get there.
The problem
Context is the tax every LLM application pays. Prompts grow, histories accumulate, retrieved passages pile up, and the bill scales with all of it — while the answer usually depended on a fraction of what was sent.
How it works
- Compresses the context an application sends to a model, targeting a 50–90% token reduction while holding the answers steady.
- Local-first by design: compression happens in the application’s own process rather than through a hosted service, so no additional party sees the prompt.
- Published as an Apache-2.0 Python package so it can sit inside an existing pipeline rather than replacing it.
Why it exists
The cheapest token is the one you never send. This is the same instinct as the research — efficiency as an engineering discipline, not an afterthought.
More work
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.