Zero-Mem: Zero-Token Memory Operations for LLM Agents

Zero-Mem proposes that an agent memory system can eliminate LLM calls from every operation outside final question answering — what the authors call zero-token memory operations. Rather than using LLMs to summarize, abstract, or generate intermediate memory representations, Zero-Mem preserves original interaction traces as the authoritative source and builds two complementary, non-generative views over them: an entity–context graph (observed co-occurrence, not generated triples) and a temporally ordered hierarchy (turns → windows → episodes). A lightweight query profile coordinates the two views; deterministic calibration discards conflicting evidence. Only the final QA reader invokes an LLM.

Evaluated on LoCoMo (long-term conversational memory) and HotpotQA (multi-hop question answering up to 448K tokens), Zero-Mem achieves the best average F1 and BLEU-1 scores across GPT-4o-mini and Qwen2.5-14B backbones while consuming zero LLM input/output tokens for memory operations. With an identical final-QA reader and context budget, it reduces memory-operation latency by 57.6% relative to the fastest compared baseline (LightMem). Ablations confirm that both the graph and hierarchical views contribute, and that evidence closure and calibration provide additional gains — showing that structured agent memory does not require generated intermediate representations.