Reference code for my TechRxiv preprint: hierarchical latent state-space recurrence that democratises long-context LLMs on edge devices, as a transformer alternative.
The problem
Self-attention costs quadratic time and memory in sequence length. That is survivable in a datacentre and fatal on a phone — which is why long-context models are effectively a cloud-only privilege.
How it works
- Replaces quadratic attention with hierarchical latent state-space recurrence: context is carried in a compressed recurrent state rather than an all-pairs attention matrix.
- The hierarchy runs several timescales at once — fast, dense levels near the input and slow, sparse levels that hold long-range summary — so distant context survives without every token attending to every other token.
- Targets edge hardware explicitly: the design goal is long context on devices with fixed memory budgets, not another point on the datacentre scaling curve.
Why it exists
This is the reference implementation for my TechRxiv preprint, and the line of work I took to both New York stages in 2026.
More work
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.