the short version
- Memorilla compresses information from a whole document collection into 16 query-dependent memory tokens. These are placed directly in a frozen LLM's input embedding space, with no retrieved text in the prompt.
- On FactKG it reached 84.33% accuracy against 80.43% for full-context inference while using 108x fewer tokens to represent the source documents.
- It was best on five of eight benchmarks against baselines including Mem0 and MemoRAG. It does not win everywhere, and the abstract does not say which three it lost.
- The abstract reports no latency figure and no embedding storage cost. A 0.79-second vs 19-second retrieval comparison circulating on Hacker News has not been verified against the paper.
Memorilla stores documents once as reusable embeddings and gives a frozen LLM 16 learned, query-dependent memory tokens in its input embedding space, instead of retrieved text chunks. On FactKG it reached 84.33% accuracy versus 80.43% for full-context inference, while using 108x fewer tokens to represent the source documents. Against RAG-style memory baselines including Mem0 and MemoRAG, it was best on five of eight benchmarks. The abstract gives no latency or embedding storage numbers.
The paper is arXiv 2610.21375, Memorilla: Latent Semantic Memory for LLMs, by Adibvafa Fallahpour, Jure Leskovec and five co-authors. It was first submitted on 15 October 2026 and revised to v3 on 3 November 2026. The authors frame the problem this way: a frozen LLM can only use what a memory system places in its context window. Gathering and combining the relevant information for each query while keeping that context short remains hard for existing systems.
How Memorilla LLM memory differs from RAG chunks
According to the abstract, Memorilla encodes documents such as user chats or book chapters once into reusable embeddings. It then learns to retrieve query-specific information and place it in the LLM's input embedding space. The model never reads retrieved passages as text. It receives a fixed number of query-dependent memory tokens, 16 in the headline results, which gather information from the entire document collection.
Memorilla, a memory system that gathers information from an entire document collection into a fixed number of query-dependent 'memory tokens' for a frozen LLM.
Commenters on Hacker News described it as a specialized embedding of a RAG. In their reading, a model in the middle takes the agent's queries and injects its response into the main model's input stream, skipping text entirely. One commenter called this injecting a vibe rather than quotable text.
Another commenter identified a structural difference from standard RAG. Instead of reranking document vectors and keeping the top five or so, Memorilla mixes relevance-weighted information from all documents into the fixed token budget. That commenter said this could help with a problem they had hit in RAG: a fact that is not local to a single patch of text. The abstract's wording, gathering from an entire collection, is consistent with that reading, but the abstract does not describe the weighting scheme.
There is a deployment implication. The memory tokens live in the input embedding space, so a serving stack would need to accept embeddings directly rather than only token IDs. The abstract does not name the frozen LLMs the authors used. It is not yet clear which model families the trained retriever works with, or whether it must be retrained for each one.
Accuracy against Mem0 and MemoRAG: five of eight
With 16 memory tokens, Memorilla achieved the best performance on five of eight benchmarks against baselines including Mem0 and MemoRAG. The benchmarks span personalization, long-document question answering and knowledge-intensive question answering. Apart from the TextWorld agent experiment, FactKG is the only result the abstract quantifies. On FactKG, Memorilla outscored the full-context baseline while using 108x fewer tokens.
The abstract does not say which three benchmarks Memorilla lost, or by how much. One commenter on Hacker News described the results as good on concepts and less good on quotes or exact text retrieval, and linked that to the latent format. That characterization does not appear in the abstract. Teams whose workloads need verbatim recall, such as citing a clause or returning an exact identifier, should check the per-benchmark tables before choosing latent memory over text retrieval.
Another commenter framed the system as a compromise between accuracy and tokens. That commenter argued that in many situations users care more about accuracy than token count. On FactKG specifically, the reported numbers do not show that tradeoff against full context, because accuracy went up as tokens went down.
Retrieval latency is not in the abstract
The abstract reports no latency figures. One commenter on Hacker News said the authors mention a 27x speedup, with retrieval at 0.79 seconds against 19 seconds. Those two figures as quoted work out to about 24x, so the claim needs checking against the paper. The thread does not state what the 19-second baseline is.
The same commenter argued that this kind of system matters most in tight voice-based conversation loops. In that setting, recalling yesterday's conversation through an agent that reasons, makes a tool call, reads the result and reasons again is too slow. The commenter called the reported figures very promising for voice-based personal assistants.
A competing view in the thread holds that semantic search is roughly equivalent to lexical search when a frontier model is driving. On that view, a model composing Lucene or other full-text-search queries already gets semantic retrieval out of a lexical engine. The commenter who raised the voice use case agreed that for workloads that are not latency sensitive, a harness running those searches is usually enough. The sources contain no head-to-head against full-text search. The abstract's comparisons are against Mem0, MemoRAG and full-context inference.
Memory cost: 108x fewer tokens, storage unreported
On FactKG, Memorilla represents the source documents with 108x fewer tokens than full context. The number of memory tokens passed to the model is fixed, 16 in the headline results. Documents are encoded once into reusable embeddings, so encoding cost is paid at ingestion rather than per query. The abstract does not report the storage footprint of those embeddings, the compute cost of the learned retrieval step, or the training cost of the retriever, and all three are part of the real memory bill.
Text-based memory has its own cost curve, driven by what the agent chooses to write. Anthropic's post on agent memory compares two Pokémon Red runs at about 10,000 steps each. Claude Sonnet 3.7 wrote 413 memory files across 225 folders and earned one badge. Claude Fable 5 wrote nine files in two folders plus a single top-level global.md and earned four badges.
Anthropic says Sonnet 3.7's detailed record kept going out of date because the game state changes constantly. As it progressed, it had to spend more time and resources combing through its memories rather than playing. Memorilla's fixed token budget caps what reaches the context window. The abstract does not say whether its stored embeddings are ever pruned or updated.
TextWorld agents wandered less: 18.6% to 24.2%
The authors also test continual learning, where the document count, here the agent's past turns, grows throughout the trajectory. In a TextWorld environment, Memorilla reduced the agent's tendency to wander between rooms it had already explored. It raised average success from 18.6% with full-history in-context learning to 24.2%. The abstract does not give the number of episodes, the trajectory lengths or the variance.
This places Memorilla between categories that Anthropic's post keeps separate. Anthropic defines agent memory as knowledge retained between sessions and saved as context the agent can read later. It excludes weight changes, in-context learning and compaction. Memorilla leaves weights frozen but replaces readable notes with learned embeddings.
One commenter on Hacker News described three layers. Prompting updates what information the model reasons about, continual learning updates how it reasons, and embedding injection sits in the middle. For engineers, the consequence is that Memorilla's memory is held as embeddings rather than text a person can read.
Anthropic's Continual Learning Bench (CL-Bench) 1.0 results show what readable memory allows. Claude Opus 5 wrote entries containing a rule, how it knew the rule was true, answers to earlier questions and warnings about traps it had found. The Memorilla abstract does not describe any way to audit or edit what it has stored.
Questions this raises
How does Memorilla differ from standard RAG?
Memorilla encodes documents once into reusable embeddings and places a fixed number of query-dependent memory tokens, 16 in the headline results, directly into a frozen LLM's input embedding space. The model never reads retrieved passages as text, and the tokens gather information from the entire document collection rather than a top-k set of chunks.
Which benchmarks did Memorilla lose against Mem0 and MemoRAG?
The abstract says Memorilla was best on five of eight benchmarks but does not name the three it lost or by how much. A Hacker News commenter described it as stronger on concepts than on exact quotes, so teams needing verbatim recall should check the per-benchmark tables.
Does the Memorilla paper report retrieval latency?
The abstract gives no latency figures. A Hacker News commenter cited a 27x speedup with retrieval at 0.79 seconds against 19 seconds, but those figures work out to about 24x, so the claim needs checking against the paper.
Spotted an error? Email samir.s.sengupta@gmail.com and I will correct it and date the change at the top of the note. How these notes are researched, drafted and checked is in the editorial policy.