EngramEdit Rewrites LLM Facts by Editing Conditional Memory

October 9, 2026Provided by Utku Ege Tuluk

On October 7, 2026, researchers from The Hong Kong Polytechnic University, Hangzhou Diagens Biotechnology, and the University of Science and Technology of China posted EngramEdit (arXiv:2610.10533), a method that revises facts inside a large language model by editing only its conditional memory — the n-gram lookup tables of an Engram-style architecture — while leaving the Transformer backbone frozen. Knowledge editing has long been framed as a trade-off: change a fact and you either fail to stick, or you damage knowledge the model never touched. EngramEdit’s claim is that giving memory its own addressable surface separates the two.

Advanced

Conditional memory in an LLM, evidence from disabling it, and the EngramEdit update pipeline. Image credit: EngramEdit project page (arXiv:2610.10533)

Diagram showing conditional memory in an LLM, evidence from disabling Engram memory, and an overview of knowledge updates with EngramEdit — Conditional memory in an LLM, evidence from disabling it, and the EngramEdit update pipeline. Image credit: EngramEdit project page (arXiv:2610.10533)

Conditional Memory, in Brief

Conditional memory was introduced by DeepSeek in Conditional Memory via Scalable Lookup (arXiv:2601.07372, January 2026). The idea is to complement Mixture-of-Experts (MoE), which scales capacity by routing computation, with a module called Engram that scales capacity through static lookup. At each token position, the model uses the sequences of words ending there — 2-grams, 3-grams, 4-grams — to fetch learned embeddings in O(1) time, and those embeddings feed into the Transformer’s hidden states. DeepSeek scaled Engram to 27 billion parameters and reported gains that ran beyond retrieval, including BBH +5.0 and Multi-Query NIAH rising from 84.2 to 97.0.

That separation is what makes editing plausible. If factual associations live in the lookup tables rather than the weight matrices, an editor could rewrite a fact by touching memory instead of surgery on the backbone. The complication, the new paper notes, is that a single fact surfaces through many phrasings that activate different n-grams, while frequently reused n-grams are shared by unrelated facts — so a naive edit either misses paraphrases or bleeds into other knowledge.

How EngramEdit Works

The authors describe a three-stage pipeline that runs on Meituan’s LongCat-Flash-Lite, an open-weight MoE model that carries conditional memory:

Compute targets. For each edit, EngramEdit generates additional expressions of the fact (four by default) alongside the original prompt, then optimises a temporary perturbation shared across their memory representations so the model predicts the revised fact under every phrasing.
Map expressions to memory. Each target representation is traced to the n-grams it activates, captured in a mapping matrix that records which embeddings are shared across expressions and across edits.
Update memory jointly. The method solves for one update per embedding so that all expressions and edits match their targets at once, subject to reuse-based regularisation that penalises large changes to short or frequent n-grams more strongly.

EngramEdit computes target representations, maps expressions to shared n-gram embeddings, and jointly allocates regularised updates. Image credit: EngramEdit project page (arXiv:2610.10533)

Diagram of EngramEdit's target computation, expression-to-memory mapping, and joint regularised memory update with its objective function — EngramEdit computes target representations, maps expressions to shared n-gram embeddings, and jointly allocates regularised updates. Image credit: EngramEdit project page (arXiv:2610.10533)

What the Numbers Show

Evaluated after 2,000 sequential edits on the CounterFact and ZsRE benchmarks, EngramEdit posts near-perfect Efficacy and, more tellingly, far higher Generalization — success on held-out paraphrases that were never used during editing. On CounterFact it records Efficacy 99.5, Generalization 97.0, Specificity 85.2, and a Utility of 93.9, against MoEEdit’s 67.1 Generalization and 76.6 Utility. On ZsRE it reaches 97.3 Efficacy and 93.7 Generalization versus UnKE’s 86.9 and 79.0. Because several baselines already hit ~99 Efficacy, the authors argue the differentiator is recall through unseen wording, not success on the prompt that was edited.

On MQuAKE, which tests whether updated facts can be used in multi-hop reasoning, the margin widens. Under chain-of-thought prompting EngramEdit reaches nearly three times the accuracy of the strongest baseline and leads across hop counts up to four hops. The paper attributes this to intermediate reasoning steps re-activating the updated embeddings as the chain unfolds.

MQuAKE results with direct and chain-of-thought prompting, including accuracy by hop count. Image credit: arXiv:2610.10533

Chart of multi-hop accuracy on MQuAKE with direct and chain-of-thought prompting, including accuracy by hop count — MQuAKE results with direct and chain-of-thought prompting, including accuracy by hop count. Image credit: arXiv:2610.10533

Preservation is the other half of the claim. Across 5,000 sequential CounterFact edits, the model retains over 96% of its pre-edit mean F1 on six general-ability tasks, with the largest decline on MRPC, a sentence-equivalence task the authors explain can activate different n-grams across its two sentences. An ablation is pointed: removing joint allocation drops Generalization to 72.8 on CounterFact, and the authors report that more than 92% of new errors on unrelated queries involve activation of updated embeddings — evidence that collateral damage routes through shared memory.

Mean F1 during sequential editing and task-level changes after 5,000 edits, relative to pre-edit scores. Image credit: arXiv:2610.10533

Charts of mean general-capability F1 during sequential editing and task-level F1 changes after 5,000 edits — Mean F1 during sequential editing and task-level changes after 5,000 edits, relative to pre-edit scores. Image credit: arXiv:2610.10533

Limitations and What This Means

The results are tied to one architecture and one benchmark suite. EngramEdit only works where conditional memory exists, so it does not apply to the standard Transformer LLMs that dominate deployment today, and the general-capability evaluation is a compact six-task slice rather than a full model evaluation. Specificity on ZsRE sits near the pre-edit level rather than above it, and the authors dedicate an appendix to arguing that higher Specificity scores from baselines partly reflect newly correct answers rather than better preservation — a nuance worth watching as independent groups replicate the work.

What the paper does demonstrate is that a scaling component and an editing interface can be the same object. If conditional memory keeps spreading as a cheap way to add capacity, an editable memory layer would let a deployed model be corrected — for a factual error, a stale figure, a changed policy — without retraining or weight surgery. The authors release code under the MIT license and frame conditional memory as “an editable knowledge interface,” a claim that now has benchmarks behind it.

Related Coverage

Meituan Open-Sources LongCat-2.0 — background on the LongCat model family that EngramEdit edits.
DeepSeek V4.1-Flash: 552B Params, 890 Bytes of KV Cache Per Token — context on DeepSeek, whose Engram architecture underpins conditional memory.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory (arXiv:2610.10533)
EngramEdit project page
EngramEdit code repository (MIT license)
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (arXiv:2601.07372)
LongCat-Flash-Lite model card (Meituan)