Preprint · TechRxiv
HLST: Democratizing Long-Context LLMs on Edge Devices via Hierarchical Latent State-Space Recurrence
A transformer alternative built on hierarchical latent state-space recurrence - trading quadratic attention for recurrence that holds long context on hardware that fits in a pocket.
The problem
Self-attention costs quadratic time and memory in sequence length. That is survivable in a data center and fatal on a phone - which is why long-context models are effectively a cloud-only privilege.
The approach
- Replaces quadratic attention with hierarchical latent state-space recurrence: context is carried in a compressed recurrent state rather than an all-pairs attention matrix.
- The hierarchy runs several timescales at once - fast, dense levels near the input and slow, sparse levels that hold long-range summary - so distant context survives without every token attending to every other token.
- Targets edge hardware explicitly: the design goal is long context on devices with fixed memory budgets, not another point on the data center scaling curve.
Presented on stage
The talks below are the production side of the same work: serving models reliably and cheaply where the data is.
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.