Fallahpour and colleagues write about Memorilla, a memory system that compresses an entire document collection into a fixed set of sixteen query-dependent memory tokens injected into a frozen LLM's input embedding space, outperforming baselines like Mem0 and MemoRAG on five of eight benchmarks spanning personalization, long-document, and knowledge-intensive question answering.
- Memorilla encodes documents once into reusable embeddings, then learns to retrieve and place query-specific information directly in the LLM's input embedding space.
- On FactKG it reaches 84.33% accuracy versus 80.43% with full-context inference, while using 108x fewer tokens to represent the source documents.
- In a TextWorld agentic environment, Memorilla raises average success rate from 18.6% (full-history in-context learning) to 24.2% and reduces the agent's tendency to re-explore rooms it has already visited.
- The work is positioned as a step toward continually learning systems, where the document count grows as an agent accumulates past turns.
- Memorilla encodes documents once into reusable embeddings, then learns to retrieve and place query-specific information directly in the LLM's input embedding space.
- On FactKG it reaches 84.33% accuracy versus 80.43% with full-context inference, while using 108x fewer tokens to represent the source documents.
- In a TextWorld agentic environment, Memorilla raises average success rate from 18.6% (full-history in-context learning) to 24.2% and reduces the agent's tendency to re-explore rooms it has already visited.
- The work is positioned as a step toward continually learning systems, where the document count grows as an agent accumulates past turns.
