Sam Siavoshian

Engram Injection

Engram Injection: A Study of Temporal Validity in Parametric Language-Model Memory

First author, with Johns Hopkins. Preprint on arXiv:2610.15882, DOI 10.48550/arXiv.2610.15882. PDF. Other papers.

Engram Injection architecture: dated updates written into LoRA memory, a frozen Qwen2.5-3B backbone with LoRA memory and FiLM clock features in each layer, and a schematic old/new preference reversal
Figure 1: Engram Injection architecture. (a) Dated updates are written into LoRA memory. (b) A frozen backbone receives the question and FiLM clock features. Clock conditioning applies to every decoder layer except the last. (c) Schematic old/new preference reversal; these bars are not measured accuracies.

Abstract

A stored fact may stop being valid when a user changes a preference or an event replaces an earlier one. We study whether a language model can learn these changes in its parameters and answer correctly from only a question and the current time. Engram Injection stores facts in LoRA adapters and supplies the clock through FiLM conditioning on a frozen Qwen2.5-3B-Instruct model. We compare it with additive clock conditioning, prompting with the full dated history, and retrieving four dated records. We train six matched adapters across three seeds after fixing a sampler that could admit test-age examples into training. The training audit reports no such examples among 127,760 evaluated training inputs. On 62 pairs of questions asked before and after an update, at an elapsed age of seven days excluded from training, both adapter methods answer zero pairs correctly under the primary blinded assistant review with order code 1 held fixed. Full-history prompting answers 59/62 pairs correctly, and retrieval answers 62/62. Separate experiments show that adapters can store facts and that removable memory units can support local deletion. Combining fact selection with time-based version selection remains unsuccessful. These results separate learning a fact from learning when to use it.

Attribute selection by semantic score, version selection by update time (Boston before the update, Seattle after), local deletion of 30 memories with retained recall unchanged, and the proposed joint routing of semantic score and temporal validity
Figure 4: Attribute selection, version selection, and deletion. (a) The question selects an attribute. (b) The clock should select its valid version. (c) An earlier bank with 30 forget and 30 retain memories illustrates deletion, separately from TOFU and the joint-bank replication. The dashed panel shows the intended combination. Table 10 reports the joint system’s measured performance.

Cite

APA

Siavoshian, S., Ramadan, O., Saeed, A. K., Johnson, B. A., & Rodriguez, B. M. (2026). Engram Injection: A study of temporal validity in parametric language-model memory (arXiv:2610.15882). arXiv. https://doi.org/10.48550/arXiv.2610.15882

BibTeX
@misc{siavoshian2026engram,
  title         = {{Engram Injection}: A Study of Temporal Validity in Parametric Language-Model Memory},
  author        = {Siavoshian, Sam and Ramadan, Omar and Saeed, Amir Kashif and Johnson, Benjamin A. and Rodriguez, Benjamin M.},
  year          = {2026},
  eprint        = {2610.15882},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi           = {10.48550/arXiv.2610.15882},
  url           = {https://arxiv.org/abs/2610.15882}
}