Preprint · TechRxiv
HLST: Democratizing Long-Context LLMs on Edge Devices via Hierarchical Latent State-Space Recurrence
A transformer alternative built on hierarchical latent state-space recurrence — trading quadratic attention for recurrence that holds long context on hardware that fits in a pocket.
A transformer alternative built on hierarchical latent state-space recurrence — trading quadratic attention for recurrence that holds long context on hardware that fits in a pocket.
The problem
Self-attention costs quadratic time and memory in sequence length. That is survivable in a datacentre and fatal on a phone — which is why long-context models are effectively a cloud-only privilege.
The approach
- Replaces quadratic attention with hierarchical latent state-space recurrence: context is carried in a compressed recurrent state rather than an all-pairs attention matrix.
- The hierarchy runs several timescales at once — fast, dense levels near the input and slow, sparse levels that hold long-range summary — so distant context survives without every token attending to every other token.
- Targets edge hardware explicitly: the design goal is long context on devices with fixed memory budgets, not another point on the datacentre scaling curve.
Presented on stage
Presented this line of work on stage in 2026 — see the talks above.
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.