Research area drill-down

Memory Poisoning

Papers currently mapped into this multi-agent security subarea from the merged research feed.

Active feeds: arXiv, OpenAlex, Crossref, Semantic Scholar, DBLP

0 of 36 articles selected

Showing 36 of 305 matching articles

Kernel-Managed Shared Memory for System-Wide Personalization

Merged record merged scholarly record arXiv Memory Poisoning Prompt Injection Governance and Policy

Ryan Lum, Yongfeng Zhang

Published 2026-09-09

Venue: arXiv

Open Source Record

Abstract

AI systems become more useful when they can adapt to the people using them, but in multi-agent systems, useful context learned by one agent often remains unavailable to others. We present kernel-managed shared memory, a system-level abstraction in which specialized agents write structured, tagged memories while the agent-system kernel, not individual agents, governs retrieval, privacy enforcement, and prompt injection. We implement and evaluate this design on AIOS and compare it against three alternatives across three assistant models (GPT-4o, Llama-3.1:8B, Qwen-2.5:7B) and 1,800 total trials. Against an unmanaged external memory backend (Mem0) using identical underlying storage, kernel-managed retrieval and injection improve personalization scores by 2.4-4.0 points on a 5-point scale (e.g., 1.05 to 4.69 profile usage on GPT-4o), with every comparison significant at p < 10^-18. Against standard retrieval-augmented injection, gains are similarly large and consistent across all three models. Against full, unfiltered context concatenation, a soft ceiling on available context rather than on response quality, kernel-managed injection statistically matches performance on two of three models and shows a small, model-specific deficit on the third, while using substantially shorter prompts: end-to-end latency is 15-61% lower across all three models, with corresponding reductions in per-call token usage and inference cost. These results indicate that centralizing memory management in the agent-system kernel, rather than leaving retrieval and privacy enforcement to individual agents, delivers most of the personalization benefit of unconstrained context at a fraction of its cost.

Bullet Summary

  • The paper addresses personalization challenges in multi-agent AI systems, where useful learned context by one agent is often inaccessible to others, limiting overall adaptability.
  • It introduces kernel-managed shared memory, a system-level abstraction where the agent-system kernel centrally manages memory retrieval, privacy enforcement, and prompt injection, rather than dispersing these tasks across individual agents.
  • Specialized agents write structured and tagged memories, while the kernel handles memory visibility, write ordering, identity resolution, retrieval ranking, formatting, and injection to ensure consistent and private personalization context.
  • Experimental evaluation on AIOS with three assistant models (GPT-4o, Llama-3.1:8B, Qwen-2.5:7B) over 1,800 trials shows that kernel-managed memory significantly outperforms unmanaged external memory and standard retrieval-augmented generation in personaliza...
  • Kernel-managed shared memory achieves comparable personalization performance to full unfiltered context concatenation but with 15-61% lower end-to-end latency, reduced token usage, and inference costs.

MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI

arXiv preprint arXiv Memory Poisoning Trust and Identity Governance and Policy

Ayan Roy, Kaustuvi Basu

Published 2026-09-08

Venue: arXiv

Open Source Record

Abstract

Agentic AI systems with persistent memory introduce a distinct attack surface known as memory poisoning, in which adversarially crafted content is stored in long-term memory and subsequently influences future agent behavior. Such attacks can suppress security alerts, facilitate privilege escalation, alter trust relationships, or override security policies without modifying the underlying model weights or system prompts. To address this threat, we present MemSentry, a formal, configuration-driven framework that intercepts proposed persistent-memory writes and produces deterministic Accept, Review, or Quarantine decisions. MemSentry evaluates each write by jointly considering source trust, semantic risk, attack radius over a component-dependency DAG, access risk, and a signed security-state delta that captures whether an operation weakens or strengthens the system's security posture. We instantiate the protected environment using a 20-asset random dependency DAG and a 10 x 20 user access-control matrix, and evaluate the framework over 1,000 GPT-4-generated scenarios using a stratified 70/30 train/test split. Semantic classification is treated as a pluggable component rather than a primary contribution, and we compare four representative approaches: rule-based Regex, TF-IDF+SVM, SBERT+LR, and SetFit. SBERT+LR achieves the best overall performance with 91.7% accuracy and a 0.908 macro-F1 score, while all four methods detect 100% of external quarantine-class threats. For verified insiders, where source trust is maximal (T = 1), MemSentry does not automatically quarantine suspicious operations but instead escalates potentially dangerous writes for human review, making semantic classification important for accurately capturing insider intent.

Bullet Summary

  • Agentic AI systems with persistent memory are vulnerable to a novel attack vector called memory poisoning, where adversarially crafted content stored in long-term memory manipulates future agent behavior without altering underlying model weights or prompts.
  • MemSentry is a formal, configuration-driven framework designed to intercept persistent-memory writes, making deterministic Accept, Review, or Quarantine decisions based on comprehensive risk assessment factors including source trust, semantic risk, attack r...
  • The framework models the system environment as a directed acyclic graph (DAG) representing component dependencies with assigned criticality, and employs a user access-control matrix to realistically simulate operations; experiments involve 1,000 GPT-4-gener...
  • Semantic classification is a pluggable component in MemSentry; evaluated methods include rule-based Regex, TF-IDF+SVM, Sentence-BERT with Logistic Regression (SBERT+LR), and SetFit. SBERT+LR achieved the best performance with 91.7% accuracy and a macro-F1 s...
  • MemSentry quarantines suspicious writes from external sources but escalates potentially harmful writes from verified insiders for human review, highlighting the challenge of detecting insider threats semantically and the importance of accurate semantic clas...

BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents

arXiv preprint arXiv Trust and Identity Memory Poisoning Governance and Policy

Yanhong Qian, Xuanying He, Qingguo Meng, Shihao Ding, Xingbo Dong, Zhe Jin

Published 2026-09-08

Venue: arXiv

Open Source Record

Abstract

KV cache is evolving from a serving optimization into an external memory substrate for long-term LLM agents. In a shared multi-user deployment, however, reusable KV blocks introduce a missing access-control question: semantic relevance alone cannot determine whether a memory block is authorized for the current physical user. We propose Bio-MemArt, a biometric-aware KV-cache memory framework for multi-user LLM agents. Bio-MemArt attaches a normalized biometric template to each stored KV memory block, filters the shared memory pool with the current user's biometric probe, and then runs the original MemArt retrieval and KV reuse pipeline only inside the authorized candidate pool. This design preserves latent-space retrieval, direct cache reuse, and decoupled position encoding while adding physical-user access control to shared KV memory. We evaluate Bio-MemArt under Owner and Non-owner query conditions on long-term dialogue QA with face and palmprint benchmarks. Across face benchmarks, the average owner and non-owner biometric success rates are 95.71% and 0.86%; across palmprint benchmarks, they are 97.60% and 2.00%. In the efficiency study, average prefill tokens drop from 18,781.96 under full-context prompting to 28.57 with Bio-MemArt, showing that biometric gating preserves the low-token operating regime of KV-cache memory.

Bullet Summary

  • Introduces Bio-MemArt, a biometric-aware KV cache memory framework designed to add physical-user access control in multi-user large language model (LLM) agent deployments.
  • Each key-value (KV) memory block stores a normalized biometric template (such as face or palmprint embeddings) to authenticate users before memory retrieval and reuse, ensuring memory blocks are accessed only by their rightful owner.
  • The framework applies biometric gating by filtering the shared KV memory pool using the current user's biometric probe, thereby creating an authorized candidate memory pool for semantic retrieval via existing MemArt pipelines.
  • Evaluations on face and palmprint biometric benchmarks show high owner authentication rates (~95-98%) and very low unauthorized (non-owner) access (~1-2%), effectively safeguarding private memory reuse.
  • Bio-MemArt maintains the efficiency advantages of KV cache memory, reducing prefilled tokens drastically (from about 18,782 to 29) compared to full-context prompting, without impacting runtime performance.

Personalizing LLM Agent Memory Using Biometrics

arXiv preprint arXiv Trust and Identity Memory Poisoning Benchmarks and Evaluation

Yanhong Qian, Qingguo Meng, Shihao Ding, Xingbo Dong, Zhe Jin, Hanrui Wang, Isao Echizen

Published 2026-09-08

Venue: arXiv

Open Source Record

Abstract

Personalized memory helps LLM agents deliver stable, tailored assistance by storing and reusing user-specific data across interactions. In multi-user scenarios, however, retrieval must consider not only semantic similarity but also whether the current requester matches the identity associated with the stored memory. We propose Bio-Memory, a biometric-aware memory architecture that conditions memory retrieval on both semantic similarity and biometric matching. Built on top of A-Mem, Bio-Memory augments each atomic memory note with a biometric embedding and uses biometric matching to form the retrieval candidate pool before semantic ranking. We evaluate Bio-Memory on LoCoMo in a 10-user shared-agent setting over 7 face benchmarks and 10 palmprint protocols. Across datasets, Bio-Memory consistently separates owner and non-owner queries. Under face-based personalization, the largest average gap reaches 27.29% / 21.15% in F1 / BLEU-1 on CALFW; under palmprint-based personalization, the corresponding gap is 25.75% / 19.22% on MS_Blue. These results support biometrics as a practical control signal for personalized memory retrieval in shared environments.

Bullet Summary

  • Personalized memory in LLM agents enhances tailored assistance by storing and reusing user-specific data across interactions, but multi-user shared environments pose a challenge in correctly retrieving memories corresponding to the current user's identity.
  • Bio-Memory is proposed as a biometric-aware memory architecture that integrates biometric embeddings with each memory note, enabling retrieval conditioned on both biometric matching and semantic similarity to ensure correct user memory access.
  • Built upon the existing A-Mem system, Bio-Memory first filters candidate memory notes by biometric similarity before applying semantic ranking for relevance, effectively separating owner from non-owner queries in multi-user scenarios.
  • The system is evaluated experimentally on a 10-user shared-agent setup using seven face recognition benchmarks (e.g., CALFW) and ten palmprint recognition protocols, demonstrating significant improvements in retrieval accuracy and personalized question answ...
  • Quantitative results show that Bio-Memory achieves large performance gaps between owner and non-owner queries (e.g., up to 27.29% F1 and 21.15% BLEU-1 score improvements for face biometrics), indicating robust access control to personal memories.

Beyond Agent Harnesses: Cross-Substrate Authority for Multi-Agent Systems

Merged record merged scholarly record arXiv Governance and Policy Memory Poisoning Benchmarks and Evaluation

Yang Li, Sergey Volkov, Hai Liu, Zongsi Xu, Xiyu Chen, Tuo Zhou, Dian Shao, Hao Sun

Published 2026-09-08

Venue: arXiv

Open Source Record

Abstract

Agentic systems persist model-visible memory while mutating workspaces, while a runtime, registry, or approval service may hold authority state outside both. Identical final files can then require opposite safe actions. We call this the cross-substrate authority gap: decision- relevant authorization information resides outside the planner-visible workspace or memory state. Across two controlled mini-benchmark families, three experiments compare planner-observation augmentation with an execution-time authority check using real Git lineage, durably recorded agent execution attempts, deterministic oracles, and two model routes. Experiment 1 is a 128-cell controlled evidence ablation: authority-blind candidate evidence obtains 0/32 final semantic success, while raw receipts and a typed relation both obtain 32/32. The missing authority fact accounts for the gain; typed packaging provides no observed planning-accuracy gain over equal raw information. Experiment 2 uses 96 planning calls: workspace-visible evidence yields 12/16 unsafe publication decisions, and planning with the typed relation remains unreliable (15/32 first actions correct; 11/32 invalid or absent). Experiment 3 replays the same 32 fixed model-generated first-action intents with zero additional model calls; a deterministic execution guard prevents all six unsafe intents from becoming effects and permits all 12 valid authorized publish intents. These results position authority enforcement at the mutation boundary as the operational endpoint of memory governance.

Bullet Summary

  • The paper addresses the 'cross-substrate authority gap' in multi-agent systems, where critical authorization information exists outside the planner-visible workspace or memory, causing identical final files to require opposite safe actions.
  • It compares planner-observation augmentation with execution-time authority checks using real Git lineage, durably recorded agent executions, deterministic oracles, and model-generated plans across three controlled experiments.
  • Experiment 1 shows that authority-blind candidate evidence leads to 0/32 successful final semantic outcomes, while raw authority receipts or typed relations enable perfect (32/32) success, highlighting the necessity of including authority facts in planning.
  • Experiment 2 reveals that relying solely on workspace-visible evidence results in unsafe publication decisions in 12/16 cases and unreliable planning with typed relations, with only 15/32 correct and 11/32 invalid or missing initial actions.
  • Experiment 3 demonstrates that a deterministic execution-time enforcement guard effectively blocks all unsafe intents and permits authorized publish intents without additional model calls, confirming the need for runtime authority enforcement.

AgentLeak: Cloning Stronger LLM Agent Capabilities onto Weaker Agents Beyond Skill Stealing

Merged record merged scholarly record arXiv Prompt Injection Memory Poisoning Orchestration Risk

Xiaoting Lyu, Yuhong Wu, Yufei Han, Shichang Liu, Liang Zhang, Bin Wang, Xiaobo Ma, Wei Wang

Published 2026-09-07

Venue: arXiv

Open Source Record

Abstract

Large language model (LLM) agents increasingly achieve long-horizon tasks by combining foundation models with explicit skills and implicit procedural knowledge acquired through execution. The resulting task-solving capabilities have become valuable proprietary assets, raising a new security question: can a substantially weaker attacker-controlled agent acquire the capabilities of a stronger proprietary agent through limited black-box interaction? Existing skill-stealing attacks recover explicit skill artifacts, yet we show that artifact leakage does not necessarily transfer capability: a weaker agent may possess the same skills but still fail because it lacks procedural behaviors implicitly realized by the stronger agent. Our key insight is that the skill execution gap itself forms a leakage surface, where missing behaviors are exposed through observable differences between successful victim executions and failed attacker executions. Based on this, we present AgentLeak, a black-box capability-cloning attack that identifies capability-critical behaviors from these execution differences and incorporates them into attacker-side skills, while keeping the attacker's model, harness, and tools unchanged. Across 20 task scenarios comprising 600 instances, diverse agent systems, and multiple backbone models, AgentLeak improves task pass rates by over 40% compared with direct skill reuse and recovers more than 80% of the victim--attacker capability gap. Our findings reveal a confidentiality risk in LLM agents: protecting explicit artifacts alone is insufficient, as observable execution behavior can leak the procedural knowledge required to reconstruct proprietary task-solving capabilities in low-capability and attacker-controlled agents.

Bullet Summary

  • Large language model (LLM) agents combine explicit skills and implicit procedural knowledge to solve complex tasks, forming valuable proprietary capabilities that pose new security challenges regarding capability leakage.
  • Existing skill-stealing attacks focus on recovering explicit skill artifacts but fail to transfer actual task-solving capabilities due to missing implicit procedural behaviors crucial for successful execution.
  • AgentLeak is introduced as a novel black-box capability-cloning attack that compares execution trajectories of a strong victim agent and a weaker attacker agent to identify missing procedural behaviors and incorporate them into attacker skills without alter...
  • The method works by abstracting victim executions into workflow skills, diagnosing capability deficits via differential analysis, extracting behavioral primitives representing missing execution behaviors, and iteratively refining attacker-side skills throug...
  • Extensive evaluation across 20 diverse task scenarios and 600 instances shows AgentLeak improves task pass rates by over 40%, recovering more than 80% of the capability gap between victim and attacker agents, and outperforming baseline methods by over 35 pe...

Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

arXiv preprint arXiv Prompt Injection Memory Poisoning Agent-to-Agent Communication

Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit, Tomer Simon, Yisroel Mirsky

Published 2026-09-06

Venue: arXiv

Open Source Record

Abstract

We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.

Bullet Summary

  • Introduces a novel side-channel attack that reconstructs text generated by locally hosted large language models (LLMs) by monitoring CPU cache activity during the detokenization process, which is a fundamental component of LLM inference pipelines.
  • Combines Flush+Reload and Prime+Probe cache attack techniques to precisely detect and isolate token-dependent cache accesses, overcoming noise and collisions inherent in cache measurements.
  • Employs a clustering approach and language model-based sequence reconstruction pipeline to translate noisy cache access patterns into semantically accurate outputs, achieving high fidelity in reconstructed text.
  • Demonstrates broad applicability and vulnerability across multiple tokenizer implementations (e.g., Llama.cpp, HuggingFace Transformers), hardware platforms, inference frameworks, and LLM families, including agentic systems like OpenClaw.
  • Performs extensive evaluations on various datasets and real-world local LLM deployments, achieving up to 90% accuracy in recovering semantic content, highlighting a significant security risk for privacy in local AI systems.

Fixed-Time Integral Reinforcement Learning for Saturated Nonlinear Multi-Agent Systems Under FDI Attacks

Merged record merged scholarly record arXiv Orchestration Risk Memory Poisoning Benchmarks and Evaluation

Tien Dat Vu, Minh Doan

Published 2026-09-05

Venue: arXiv

Open Source Record

Abstract

The leader-follower formation control problem is investigated for nonlinear multi-agent systems with unknown dynamics, external disturbances, and false data injection (FDI) attacks on actuator channels. The problem is formulated as a zero-sum differential game and solved using the Integral Bellman-Isaacs approach. To address input saturation constraints, a non-quadratic control cost function is incorporated into the optimization problem, leading to a bounded control law. Furthermore, this paper proposes a cost function construction method and develops a critic learning law, which together guarantee the practical fixed-time stability of the system while overcoming the limitations of existing fixed-time reinforcement learning formulations. Finally, the practical fixed-time convergence of both the critic weight estimation error and the leader-referenced formation tracking error to bounded residual sets is rigorously proven. Simulation results demonstrate the effectiveness of the proposed method under external disturbances, FDI attacks, and input constraints.

Bullet Summary

  • The paper addresses leader-follower formation control for nonlinear multi-agent systems with unknown dynamics, external disturbances, and false data injection (FDI) attacks on actuator channels, modeling the problem as a zero-sum differential game.
  • A non-quadratic control cost function incorporating actuator input saturation constraints is proposed, resulting in bounded control laws that respect physical input restrictions.
  • A novel cost function construction method and a critic learning law are developed, guaranteeing practical fixed-time stability and fixed-time convergence of both the critic weight estimation error and the formation tracking error to bounded residual sets.
  • The approach employs the Integral Bellman-Isaacs methodology with a critic neural network leveraging integral Bellman-Isaacs residuals, enabling data-driven policy learning without requiring explicit knowledge of system drift dynamics.
  • Experience replay with finite data windows and normalization is utilized to improve learning efficiency and to avoid persistent excitation conditions, supporting a fixed-time critic weight update law with leakage ensuring bounded approximation error.

Structurally Close, Temporally Distant: Measuring Security Exposure in Long-Horizon LLM Agents

arXiv preprint arXiv Memory Poisoning Prompt Injection Agent-to-Agent Communication

Md Jafrin Hossain, Nur Al Hasan Haldar

Published 2026-09-05

Venue: arXiv

Open Source Record

Abstract

Long-horizon LLM agents interact with untrusted content, persistent memory, external state, and sensitive tools. Existing analyses often characterize attacks by the number of execution steps between malicious input and a downstream action. We show that temporal remoteness can overstate security separation in stateful agents. We introduce a provenance-aware execution graph linking agent events through deterministic state, identifier, and tool provenance, and define \emph{influence distance} $\DI$ as the shortest structural path from an untrusted source to a sensitive action. We compare it with \emph{sequence distance} $\DT$, the shortest injection--sink path in the ordered trajectory. Since the influence graph contains every sequence edge, $\DI \leq \DT$; $\Gap=\DT-\DI$ measures the separation hidden by step count. Across 454 injection--sink pairs from 360 long-horizon AgentDojo trajectories over OpenAI's \texttt{gpt-4o-mini} and \texttt{gpt-4o} and Claude's Haiku 4.5 and Sonnet 4.6, $\Gap>0$ for 96.9% of pairs, with a median gap of 9 hops; 91.0% remain decoupled after removing the largest provenance-only edge class. On AgentDojo's banking suite, 33.8% of 231 pairs from 377 trajectories decouple through different provenance mechanisms. Among 274 OpenAI pairs, $\Gap$ does not independently predict attack success after controlling for $\DT$, attack family, and backend ($β_{\Gap}=0.066$, $p=.088$). At matched thresholds $k=2,3$, a deterministic $\DI$-based pre-execution gate blocks five attack sinks missed by a sequence-only gate with no additional benign blocking, although the paired gain is not significant ($p=.0625$). Execution structure therefore reveals proximity hidden by step count and can support targeted runtime intervention. We measure candidate influence pathways rather than causal attribution.

Bullet Summary

  • Long-horizon large language model (LLM) agents face security challenges when interacting with untrusted inputs, persistent memory, external states, and sensitive tools, which can enable indirect prompt injection attacks.
  • Traditional security analyses using temporal remoteness—measuring the number of execution steps between malicious input and sensitive actions—can overstate actual security by neglecting structural relationships.
  • The authors introduce a provenance-aware execution graph and define influence distance (DI) as the shortest structural path connecting untrusted sources to sensitive actions, contrasting it with sequence distance (DT) based on execution order.
  • Empirical results across 454 injection–sink pairs from multiple LLMs (OpenAI's GPT-4o-mini, GPT-4o, and Anthropic's Haiku 4.5, Sonnet 4.6) reveal that the influence distance is often significantly smaller than the sequence distance, indicating hidden struct...
  • The structural gap (Gap = DT - DI) quantifies how step count metrics can obscure actual closeness, with 96.9% of evaluated pairs showing a positive gap and a median gap of nine hops.

Forgetting Without Restarting: Execution-State Unlearning for Stateful LLM Agents

arXiv preprint arXiv Memory Poisoning Prompt Injection

Chao Yao, Yangbo Wei, Zhen Huang, Junhong Qian, Chenle Chen, Shaoqiang Lu, Chen Wu, Lei He

Published 2026-09-04

Venue: arXiv

Open Source Record

Abstract

Long-running LLM agents are stateful: beyond the transcript they accrete compressed summaries, plaintext memory, pending tool plans, and, under every serving API, a KV cache. Yet today's "forget" operations delete a plaintext memory record and stop, leaving every artifact derived from the revoked information intact. We formalize execution-state unlearning: after a forget request, the agent must behave as if it had never observed the target. Modeling the runtime as a deterministic transition system, we prove that the pre-target trajectory prefix is shared with this counterfactual world for free, that the post-target suffix is irreducibly tainted without token-level attribution, and that exact unlearning requires at least $T-τ+1$ recomputed transitions, where $τ$ is the target's injection step. Provenance-Guided Selective Replay attains this bound as a cross-layer contract spanning prompt, compressed memory, and cache: a provenance graph locates the injection point, checkpoint restoration reduces to cropping the KV cache, and sanitized replay regenerates the counterfactual suffix. Audited with elicitation, stochastic, and string-free behavioral tests across three agent suites, nine baselines, and three model families, memory deletion leaves leakage unchanged, instruction-based forgetting collapses under elicitation (Leak@probes = 1.00), and source redaction still acts on a revoked preference in 80% of episodes, while selective replay is indistinguishable from a full reset at up to 9x fewer recomputed tokens.

Bullet Summary

  • Long-running LLM agents maintain complex, multilayered state involving transcripts, compressed summaries, plaintext memory, pending tool plans, and KV caches, making forgetting operations challenging.
  • Existing forget methods only delete explicit plaintext memory records but fail to erase derived artifacts such as summaries and cached states, resulting in persistent leakage of revoked information.
  • The paper formalizes 'execution-state unlearning', modeling the agent's runtime as a deterministic transition system, and defines the goal as making the agent behave as if it never observed the target information.
  • Theoretical analysis establishes that prefix states before the target injection point can be reused ('free prefix'), but exact unlearning requires replaying and recomputing all transitions after the injection step due to tainted suffix states.
  • A novel method, Provenance-Guided Selective Replay, efficiently implements unlearning by using a provenance graph to identify the injection point, restoring checkpoints, cropping the KV cache, and replaying sanitized inputs ignoring the target data.

The Double-Edged Sword of AI Pair Programmers: A Systematic Literature Review of Security Vulnerabilities in AI-Generated Code and Agentic Development Environments

Merged record merged scholarly record OpenAlex Prompt Injection Memory Poisoning Governance and Policy

Mahmoud E. Farfoura, Mohammad A. K. Alia, Ibrahim Mashal, Adnan A. Hnaif, Mahmoud Odeh

Published 2026-09-04

Venue: Journal of Sustainable Smart Systems in Education & Environment

DOI: https://doi.org/10.66823/2egmd793

Open Source Record

Abstract

The role of AI pair programmers has expanded from local code completion to active participation in the development environment. Contemporary tools can interpret repository context, edit multiple files, call package managers, execute terminal commands, and communicate with external services. This review synthesizes security evidence concerning GitHub Copilot, ChatGPT-based coding, code large language models, Cursor-style agentic editors, command-line coding agents, and Model Context Protocol ecosystems. A protocol-driven search, supplemented by backward and forward snowballing, covered literature and technical evidence available through 12 July 2026. The verified corpus comprised 216 verified records, including peer-reviewed studies, preprints, benchmarks, standards, and clearly identified technical disclosures. The evidence supports useful roles in vulnerability discovery and repair, test generation, and secure-coding guidance, but it also documents recurring injection flaws, unsafe memory and file handling, weak cryptography, authentication errors, secret exposure, hallucinated dependencies, and incomplete patches. Whether these weaknesses persist depends partly on human factors, including expertise, prompt framing, review effort, automation bias, and the authority users assign to the assistant. Most generated-code defects remain familiar CWE classes. Agentic systems add orchestration-level concerns: indirect prompt injection, context poisoning, tool and protocol supply-chain attacks, permission amplification, approval spoofing, cross-file persistence, and autonomous execution. We synthesize these findings in a unified taxonomy, an evidence-based threat model, a lifecycle security model, and a continuous-assurance architecture built on structured context, least privilege, sandboxing, provenance, conventional SAST/SCA/secret scanning, security tests, and mandatory human authorization for high-impact actions. Without such governance, faster production may be offset by accumulating validation debt and downstream incident risk.

Bullet Summary

  • AI pair programmers have evolved into agentic environments capable of multi-file edits, terminal commands, and interacting with external services, enhancing development productivity but introducing new security challenges.
  • The review analyzes 216 verified records on security vulnerabilities related to AI-generated code and agentic IDEs, identifying classic and emerging threats such as injection flaws, weak cryptography, prompt injection, and context poisoning.
  • Human factors like expertise, prompt design, and review diligence critically influence security outcomes; automation bias and confidence in AI outputs can increase risk if not properly managed.
  • AI-generated code commonly exhibits known vulnerability classes (e.g., CWE categories) but agentic orchestration adds complex risks like permission amplification, approval spoofing, and persistent malicious context.
  • Large language models assist in vulnerability detection and repair, offering benefits over static analyzers, yet fully automated repair is still unreliable and requires human validation.

Inferring Hidden User Models from the Behavior of Personalized LLM Agents

arXiv preprint arXiv Memory Poisoning Benchmarks and Evaluation

Haoyang Li, Yaxin Xiao, Qingqing Ye, Huadi Zheng, Haibo Hu

Published 2026-09-03

Venue: arXiv

Open Source Record

Abstract

Recent personalized LLM agents increasingly transform information retained in memory into compressed or structured representations, which we call user models, to guide later decisions. When source wording is removed from the state reachable through the ordinary interface, these models are commonly treated as more privacy-preserving because direct memory-extraction attacks lose the text they target. Yet we argue that user models expose a new attack surface because an attacker can still recover the private information from the personalized choices they shape, even when source records and backend state remain inaccessible. We therefore introduce UMPeek, a black-box attack based on hypothesis-guided adaptive probing to infer such hidden user model. It forms hypotheses from choices left open by a request, switches among ordinary follow-up tasks, and retains only claims supported and not contradicted by visible behavior. We conduct an extensive benchmark evaluation across diverse personalization tasks and user-model backends against existing attacks. We further validate UMPeek in real-world systems using information confirmed to be retained, and we evaluate defenses against its adaptive probing. Overall, UMPeek outperforms existing attacks in both benchmark and real-world comparisons and continues to recover user information under response-level defenses, showing that keeping records and backend state inaccessible does not guarantee semantic privacy when retained information shapes visible behavior.

Bullet Summary

  • Personalized LLM agents use compressed user models derived from memory to guide decisions, believed to enhance privacy by omitting source wording and backend access.
  • Despite inaccessible backend states and omitted source text, private user information can be inferred from personalized agent behaviors, representing a novel attack surface.
  • UMPeek is a black-box attack using hypothesis-guided adaptive probing that formulates and tests behavioral hypotheses through ordinary follow-up tasks to infer hidden user models.
  • Extensive experiments across multiple benchmarks, personalization backends, and real-world systems show UMPeek significantly outperforms seven baseline attacks in semantic recovery and behavior prediction.
  • UMPeek relies on analyzing personalization-dependent decisions across multiple tasks, adaptively selecting requests to maximize evidence for or against candidate semantic claims about the user.

LLM-Based Autonomous Agents: A Systematic Review and Critical Synthesis of Architectural Paradigms, Memory Models, Planning Strategies, And Operational Limitations

Merged record merged scholarly record OpenAlex Benchmarks and Evaluation Agent-to-Agent Communication Memory Poisoning

Chukwuemeka Christiantus Ndubuisi

Published 2026-09-03

Venue: INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND MATHEMATICAL THEORY E-ISSN

DOI: https://doi.org/10.56201/ijcsmt.vol.12.no4.2026.pg239.255

Open Source Record

Abstract

Large Language Models (LLMs) have enabled the emergence of autonomous AI agents capable of reasoning, planning, tool use, and iterative decision-making. Despite rapid development, the field remains architecturally fragmented, with limited conceptual clarity regarding memory integration, planning mechanisms, and operational reliability. This study presents a systematic review and critical synthesis of LLM-based autonomous agents, focusing on architectural paradigms, memory models, planning strategies, and real-world deployment constraints. Using a structured review approach, this study examines existing LLM based agent systems across key design components to uncover common patterns, differences in implementation, and recurring structural weaknesses. The review reveals persistent and structurally significant challenges across all four dimensions: long-horizon reasoning stability degrades as task length increases; memory consistency is undermined by retrieval noise, embedding drift, and summarisation errors; tool alignment failures propagate errors across modular pipelines; and evaluation standardisation remains insufficient to support reliable cross-paper comparison. A consistent cross-paradigm finding emerges: autonomy and reliability trade off systematically as agent complexity increases, with current systems achieving capability gains through heuristic design rather than principled theoretical foundations. Based on this synthesis, the review proposes a consolidated analytical framework that maps common structural elements and trade-offs across reviewed systems, and outlines a research agenda directed toward formalised agent architectures, memory consistency guarantees, verified planning algorithms, standardised reliability metrics, and benchmark frameworks adequate for long-horizon, real-world evaluation conditions.

Bullet Summary

  • The paper addresses the fragmentation and conceptual ambiguity in the design of LLM-based autonomous agents, focusing on architectural paradigms, memory models, planning strategies, and operational limitations.
  • It systematically reviews 142 empirical studies (2020-2026) on LLM-based agents to identify common design patterns, challenges, and structural weaknesses.
  • Agent architectures are framed as closed-loop systems with five subsystems (Perception, Brain, Memory, Action, Collaboration) formalized under a Partially Observable Markov Decision Process (POMDP).
  • Key challenges identified include degradation in long-horizon reasoning, memory consistency issues due to retrieval noise and embedding drift, tool alignment failures, and insufficient evaluation standardization.
  • Memory in agents spans in-context, external long-term (e.g., vector stores, knowledge graphs), parametric, and episodic forms, with current systems excelling at some but none fully mastering cognitive memory integration.

CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents

arXiv preprint arXiv Memory Poisoning Benchmarks and Evaluation

S M Asif Hossain, Ruksat Khan Shayoni, Md Kishor Morol

Published 2026-09-02

Venue: arXiv

Open Source Record

Abstract

Personalized language agents use persistent memory to adapt to users over time, but the same mechanism creates an attack surface. When new information conflicts with stored preferences, an agent must distinguish genuine preference drift from temporary context shifts, ambiguity, or adversarial memory poisoning. We formulate this problem as a continuous-time partially observable decision process over a latent user state and show why rules based only on recency and provenance are insufficient. CAPTURE addresses this ambiguity with a neural differential-equation belief tracker, a multi-timescale memory ledger, uncertainty-triggered clarification, and counterfactual auditing of cited memories. On 480 held-out episodes from 96 users, CAPTURE achieves a 71.5% win rate, compared with 69.3% for an identically supervised baseline and 66.1% for the strongest heuristic baseline. It limits fixed-policy poisoning success to 11.5% while accepting 83.5% of genuine preference updates. Under an adaptive attacker with access to the released weights, attack success rises to 24.7%, exposing a real adaptation-security tradeoff. We further evaluate the frozen system zero-shot on an independently constructed benchmark and replay longitudinal interaction histories from 40 users collected over two to three weeks. These results suggest that modeling preference authenticity explicitly can improve both personalization and robustness in memory-augmented LLM agents.

Bullet Summary

  • Personalized language agents use persistent memory to adapt to user preferences over time but face vulnerabilities from adversarial memory poisoning that mimics genuine preference changes.
  • The key challenge, termed preference-authenticity ambiguity, involves distinguishing genuine user preference drift from temporary context shifts, ambiguity, or malicious inputs corrupting memory.
  • CAPTURE formulates this problem as a continuous-time partially observable decision process over a latent user state and employs a neural differential-equation-based belief tracker to model and infer preference authenticity.
  • The method integrates a multi-timescale memory ledger to record beliefs with varying persistence and decay, an uncertainty-triggered clarification mechanism to ask users for disambiguation, and counterfactual auditing to verify memory citations causally.
  • Experimental evaluation on 480 synthetic user episodes and 40 real users' longitudinal data shows CAPTURE achieves a 71.5% win rate, outperforming baselines in both personalization accuracy and robustness against memory poisoning attacks.

AgentShield: A Zero-Trust Runtime Guardrail Architecture for Autonomous Multi-Agent AI Systems with Bidirectional Context Synchronization

Merged record merged scholarly record OpenAlex Prompt Injection Memory Poisoning Trust and Identity

Nandhakumar Murugan

Published 2026-09-02

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.22259021

Open Source Record

Abstract

The rapid migration of Large Language Models (LLMs) from conversational interfaces to autonomous multi-agent software engineering systems has exposed profound security vulnerabilities. When autonomous agents operate across heterogeneous topologies—spanning cloud-hosted reasoning engines and local execution environments—they are acutely vulnerable to indirect prompt injection, tool-call hijacking, privileged command escalation, and memory poisoning. Conventional boundary defenses (such as input sanitizers and prompt wrappers) fail to address lateral privilege escalation between collaborating agents. To resolve this critical architectural vulnerability, this paper introduces AgentShield, a zero-trust runtime verification and guardrail framework designed for decentralized multi-agent systems operating over the Model Context Protocol (MCP). AgentShield enforces continuous, non-bypassable policy verification across all intra-agent communications and system tool dispatches. The framework incorporates: (1) an inline bidirectional semantic interceptor that evaluates agent intents before system execution, (2) a multi-lingual token triage engine capable of detecting adversarial jailbreaks in low-resource and code-switched dialects, (3) a cryptographically signed persistent shared ledger ensuring tamper-evident state continuity, and (4) an automated capability attenuator for operating system and file operations. We evaluate AgentShield across 1,500 adversarial scenarios covering multi-step tool-injection benchmarks and real-world developer workflows. Empirical results demonstrate that AgentShield mitigates 98.4% of prompt injection and tool-escalation attacks while introducing less than 11.8ms of median runtime latency overhead. The core architecture is validated via two production-grade open-source packages released on the Python Package Index (PyPI): prema-agentshield and gemini-antigravity-bridge.

Bullet Summary

  • Rapid adoption of Large Language Models (LLMs) as autonomous multi-agent AI systems has introduced serious security vulnerabilities, particularly in heterogeneous environments combining cloud and local execution.
  • Existing boundary defense mechanisms (e.g., input sanitizers, prompt wrappers) do not effectively prevent lateral privilege escalation among collaborating agents.
  • AgentShield is proposed as a zero-trust runtime verification and guardrail framework specifically designed for decentralized multi-agent systems using the Model Context Protocol (MCP).
  • AgentShield features a bidirectional semantic interceptor that inspects agent intents prior to system execution to prevent malicious actions.
  • It includes a multi-lingual token triage engine that detects adversarial jailbreaks even in low-resource and code-switched dialects, enhancing robustness against prompt injection attacks.

Steering Severity: Single-Source Poisoning of LLM-Based CVE Severity Assessment and a Corroboration-Based Defense

Merged record merged scholarly record OpenAlex Memory Poisoning Benchmarks and Evaluation Governance and Policy

Shiddarth Dey Tusar

Published 2026-09-02

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.22253755

Open Source Record

Abstract

Large language model (LLM) agents are increasingly used to summarise cyber threat intelligence (CTI) and assign severity to disclosed vulnerabilities, typically by retrieving open-source advisories into a retrieval-augmented generation (RAG) context. I study a focused, practically realistic threat: an adversary who controls a single open-source CTI document and seeks to steer the agent’s severity assessment of a target vulnerability. Using a controlled synthetic-CVE testbed that removes answer-leakage and memorised-prior confounds, I evaluate severity steering and a training-free consistency defense across three open-weight 7–8B models (Llama 3 8B, Qwen2.5 7B, and Mistral 7B). A single poisoned source steers severity bands in the intended direction, though attack susceptibility is model-dependent (8/8 on Llama 3 8B, 7/8 on Qwen2.5 7B, 4/8 on Mistral 7B). The leave-one-out internal-consistency defense generalizes across models, achieving high detection (8/8, 7/8, 7/8) and recovery (8/8, 7/8, 8/8) with zero false positives across all models under a stated ≥4-source precondition. The inflation/deflation asymmetry observed in Llama 3 8B does not replicate on Qwen or Mistral, showing that directional asymmetry is model-specific rather than attack-intrinsic. I also find that small local models imitate CVSS output format without performing CVSS computation, which constrains evaluation to the band level. I frame this as a controlled mechanism study and outline extensions for multi-source campaigns and real-world CTI pipelines.

Bullet Summary

  • Investigates a targeted threat where an adversary controls a single open-source cyber threat intelligence (CTI) document to manipulate large language model (LLM) agents' severity assessments of vulnerabilities.
  • Utilizes a controlled synthetic-CVE testbed to eliminate confounding factors such as answer leakage and prior memorization, ensuring rigorous evaluation of poisoning effects.
  • Evaluates severity steering attacks and a training-free, leave-one-out internal consistency defense across three open-weight LLMs: Llama 3 8B, Qwen2.5 7B, and Mistral 7B.
  • Finds that a single poisoned source can reliably steer severity bands in the intended direction, but susceptibility to attack varies by model (100% on Llama 3 8B, 87.5% on Qwen2.5 7B, 50% on Mistral 7B).
  • Demonstrates that the internal consistency defense generalizes across models, achieving high detection and recovery rates with zero false positives, provided there are at least four open-source sources available.

Agent Memory Is a Surface for Endogenous Authorization Laundering

arXiv preprint arXiv Memory Poisoning Trust and Identity Governance and Policy

Tommaso Cerruti, Mika Okamoto, Ansel Kaplan Erol

Published 2026-09-01

Venue: arXiv

Open Source Record

Abstract

Long-running LLM agents rely on persistent memory to carry state across interactions, including permissions, restrictions, and revocations. When memory misrepresents this evolving authorization state, the agent's own records can grant authority that the underlying history never permitted, resulting in misaligned behavior without any external attacks. We term this failure endogenous authorization laundering, where spurious permissions written into memory lead to unauthorized actions as their provenance is washed away. We then introduce EAL-Bench, which measures how accurately persistent memory preserves evolving authorization state and whether errors propagate to downstream unauthorized actions. We evaluate five LLMs as memory writers and two as executors across procurement, cybersecurity, and finance. We find that under incremental memory updates, writers create false authority for up to 50.2% of unauthorized requests; once false authority is present, executors act on it in 98.6% of trials. Two safeguards, requiring stored permissions to be backed by valid source events, and tracking permission changes through bounded event sourcing, substantially reduce laundering, but both also reject more legitimate actions, exposing a safety-utility tradeoff. Persistent memory is therefore not merely a performance component, but a part of an LLM agent's effective authorization policy.

Bullet Summary

  • Long-running Large Language Model (LLM) agents use persistent memory to track evolving authorization states (permissions, restrictions, revocations), but inaccuracies in this memory can cause 'endogenous authorization laundering' where unauthorized permissi...
  • EAL-Bench is introduced as a novel benchmark framework that evaluates how well persistent memory preserves correct authorization states and whether errors in memory propagate into unauthorized agent actions, using domains like procurement, cybersecurity, an...
  • Empirical evaluation of multiple LLMs as memory writers and executors reveals that up to 50.2% of unauthorized requests arise due to false authorities created by memory errors; executors act on these spurious permissions in 98.6% of cases, leading to covert...
  • Memory update strategies impact error rates: incremental updates propagate earlier memory inaccuracies causing more unauthorized actions, whereas one-shot memory writing from complete histories reduces such errors.
  • Two key mitigation techniques are proposed: requiring stored permissions to be backed by valid source events (source-authority gating) and using bounded event sourcing to track permission changes through immutable logs; both significantly reduce unauthorize...

Transferable End-to-End Optimization for Indirect Long-Term Memory Poisoning in LLM Agents

arXiv preprint arXiv Memory Poisoning Prompt Injection Benchmarks and Evaluation

Chuanchao Zang, Jianing Wang, Wenyu Chen, Xiangtao Meng, Li Wang, Xinyu Gao, Zheng Li, Shanqing Guo

Published 2026-09-01

Venue: arXiv

Open Source Record

Abstract

Long-term memory can turn untrusted external content into persistent influence over an LLM agent's future decisions, creating the threat of indirect memory poisoning. A successful attack must survive a multi-stage pipeline comprising memory writing, retrieval, and utilization. Existing attacks largely rely on intra-stage optimization, optimizing individual stages in isolation while overlooking inter-stage coupling. Specifically, these stages impose different requirements on the same poisoning content, and each stage operates on the transformed output of its predecessor. Consequently, optimizing one stage may undermine the effectiveness of other stages, while upstream transformations may erase improvements intended for downstream stages. Indirect memory poisoning should therefore be viewed as an end-to-end optimization problem. Based on this insight, we present \textsc{PipePoison}, which collects fine-grained stage feedback from local shadow systems, uses chain-structured losses to identify and optimize the stage bottlenecking end-to-end success, and applies stability-calibrated stage and configuration weights to improve transferability. Across three agent frameworks and four memory mechanisms, \textsc{PipePoison} improves attack utilization rate by 19.1 percentage points. Even on fully unseen victim configurations, it outperforms the strongest baseline by 16 percentage points and remains effective under eight representative defenses.

Bullet Summary

  • Memory Agent MINJA ER-MIA Zombie Sleeper MemMorph GhostWriter PIPEPOISON WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR Langmem LangGraph 62% 88% 50% 47% 74% 36% 33% 78% 26% 43% 74% 37% 69% 89% 61% 48% 72%...
  • 4 Evaluation 4.1 Experimental Setup Agent and Memory Systems.We evaluate PIPEPOI- SONon three real-world open-source agent frameworks (GitHub stars in parentheses)—LangGraph (39.8K) [17], CrewAI (57.5K) [4], and OpenAI Agents (28.9K) [28]— combined with fou...
  • Transferable End-to-End Optimization for Indirect Long-Term Memory Poisoning in LLM Agents Chuanchao Zang1 Jianing Wang1 Wenyu Chen1 Xiangtao Meng1 Li Wang1 Xinyu Gao1 Zheng Li1,2,3* Shanqing Guo1,2,3* 1School of Cyber Science and Technology, Shandong Unive...
  • Based on whether the attacker expects the poisoning content to affect agents with a particular configuration, we consider two scenar- ios.(1) In the matched scenario , the attack is designed for a particular expected configuration and takes effect if an age...
  • Agent Prompt You are a helpful agent equipped with external tools and long-term memory.

Context Inference Attacks Without Jailbreaks

arXiv preprint arXiv Prompt Injection Memory Poisoning Agent-to-Agent Communication

Prince Jha, Samuele Poppi, Nils Lukas

Published 2026-08-31

Venue: arXiv

Open Source Record

Abstract

Agentic AI systems are increasingly deployed to process sensitive data at inference time, such as healthcare records or financial documents assembled into a hidden \emph{context} before the system answers. Prior work has studied privacy risks primarily through \emph{jailbreaking} attacks that induce models to directly disclose sensitive content, but has largely overlooked the agentic setting where the context is assembled by the agent's own tool calls. We show that the agents we evaluate remain vulnerable to hidden-context leakage despite the controls we test against them, namely an instruction not to disclose the context, logit suppression, and context dilution. For instance, a web-browsing agent answering benign user queries still carries exploitable signals about records silently loaded into its context. We introduce and formalize \emph{context-inference attacks} through a security game and evaluate three settings under decreasing attacker knowledge and increasingly indirect delivery of the context: a known context, an unknown context, and a context the agent retrieves through its own tool calls. We distinguish a grey-box setting, in which the target model is used to score observations, from black-box settings in which the attacker scores with a surrogate it controls. We further characterize how leakage varies with query budget, context size, and target-model size. A single attack carries through all three settings without modification, reaching $100\%$ ASR on small candidate sets and $63\%$ at $1024$ candidates against a known context, $78.9$ AUROC when the template and surrounding records are unknown, $92.5$ AUROC when a 14B surrogate scores a 32B target, and $81.8$ AUROC when the records arrive as an agent's retrieval returns, against chance rates of $1/|\mathcal{Z}|$ and $50$ respectively.

Bullet Summary

  • Agentic AI systems process sensitive data by assembling hidden contexts at inference time, which introduces unique privacy vulnerabilities beyond traditional jailbreak-style direct content disclosures.
  • The paper formalizes 'context-inference attacks,' which exploit subtle statistical signals in benign model outputs to infer hidden contexts without explicit disclosure or adversarial queries.
  • A unified and replay-based likelihood inference framework is introduced, allowing attackers to test hypotheses about the hidden context using benign, target-agnostic queries across varying threat models of attacker knowledge and context delivery.
  • Experiments demonstrate high effectiveness of context-inference attacks across multiple large language and vision-language models, with attack success rates exceeding 80% AUROC even under black-box settings and without adversarial instructions.
  • The attack framework is robust across three threat models: TM1 with known candidate sets and templates; TM2 with unknown templates and candidates; and TM3 where hidden context is agent-retrieved records rather than fixed text.

Understanding Stage-Wise Utility-Risk Trade-offs in LLM Agent Memory

arXiv preprint arXiv Memory Poisoning Benchmarks and Evaluation Governance and Policy

Chuanchao Zang, Zijian Cao, Xiangtao Meng, Jianing Wang, Wenyu Chen, Xinyu Gao, Li Wang, Zheng Li

Published 2026-08-31

Venue: arXiv

Open Source Record

Abstract

Long-term memory is becoming a core capability of LLM agents, enabling personalization and long-horizon interaction. However, memory mechanisms that retain, transform, or expose more information can affect both benign utility and susceptibility to memory poisoning. Existing evaluations typically measure memory utility or attack risk in isolation under fixed configurations, providing limited insight into how stage-specific design choices reshape their trade-off. We present \textsc{MemGauge}, a controllable framework that separately varies writing admission, management policy, and retrieval exposure under matched clean and poisoned conditions. Across 11 LLMs and two long-term memory benchmarks, controlled evaluations reveal three distinct profiles: a threshold-like risk transition during writing, policy-dependent local decoupling during management, and coupled growth of utility and risk during retrieval. We further apply analogous stage-level measurements to four existing memory systems and observe diagnostic associations qualitatively consistent with these profiles. These results show that targeted poisoning risk varies across memory operations and motivate stage-aware evaluation and control of LLM-agent memory.

Bullet Summary

  • The resulting stage-wise utility–risk response profiles reveal three distinct patterns.(1) Writing stage exhibits a nonlin- ear transition: as admission coverage expands along each agent’s model-specific ranking, clean utility rises and grad- ually saturate...
  • Understanding Stage-Wise Utility-Risk Trade-offs in LLM Agent Memory Chuanchao Zang1 Zijian Cao1 Xiangtao Meng1 Jianing Wang1 Wenyu Chen1 Xinyu Gao1 Li Wang1 Zheng Li1,2,3* Shanqing Guo1,2,3* 1School of Cyber Science and Technology, Shandong University 2Sta...
  • Changes at different stages may move util- ity and risk differently: one may increase both, whereas an- other may reduce risk with little utility loss.
  • Key Finding 1.Writing exhibits athreshold-like risk transition, management enablesmechanism-dependent local decoupling, and retrievalcouples utility with risk.
  • Across 11 LLMs and two long-term memory benchmarks, controlled evaluations reveal three distinct pro- files: a threshold-like risk transition during writing, policy- dependent local decoupling during management, and cou- pled growth of utility and risk duri...

Safe to Resume? Breaking Execution Continuity of Agent Execution via Rollback

Merged record merged scholarly record arXiv Orchestration Risk Memory Poisoning Trust and Identity

Guanlong Wu, Dahui Li, Ke Jiang, Jianyu Niu, Cong Wang, Yinqian Zhang

Published 2026-08-29

Venue: arXiv

Open Source Record

Abstract

AI agents are moving toward persistent, stateful execution across various applications, accumulating execution state and external effects that are costly to reconstruct after failures. Checkpoint and rollback (C/R) are becoming essential for recovery, yet their security implications remain largely unexplored. Correct rollback does not imply secure recovery: a faithfully restored checkpoint may resume an execution whose states, assumptions, and external effects never coexisted in any valid history. In this paper, we present the first systematic security study of checkpoint and rollback in existing agent systems. By examining representative agent C/R systems, we characterize the design space of existing C/R mechanisms and develop a general execution model that captures their recovery boundaries and state dependencies. From this model, we identify five fundamental failure modes spanning incomplete or inconsistent internal state, stale external dependencies, nondeterministic replay, and unrecorded external effects. We further demonstrate their security impact through three end-to-end attacks on Hermes, Cline, and LangGraph, enabling malware-verification bypass, unauthorized mail forwarding, and double payment. To systematically study these failures in practice, we develop a multi-agent analysis pipeline that reconstructs execution semantics, identifies violations of the five failure conditions, and validates them through actual rollback. Across five representative frameworks, our evaluation shows that these failures recur across heterogeneous C/R designs and stem from a common gap between the state restored by a checkpoint and the dependencies required for secure continuation.

Bullet Summary

  • AI agents increasingly rely on checkpoint and rollback (C/R) mechanisms for persistent, stateful execution, but the security implications of these recovery methods have been largely unexplored.
  • The paper presents a general execution model for agent checkpoint and rollback systems, highlighting that checkpoints restore only partial internal agent states, not the external environment, leading to possible execution continuity breaches.
  • Five fundamental checkpoint and rollback failure modes are identified: incomplete internal state coverage, inconsistent checkpoint states, stale external dependencies, nondeterministic replay, and unrecorded external effects.
  • These failure modes enable security breaches demonstrated through attacks such as malware-verification bypass, unauthorized mail forwarding, and double payments on frameworks like Hermes, Cline, and LangGraph.
  • A novel multi-agent analysis pipeline was developed to reconstruct execution semantics, detect these failure conditions, and experimentally validate them via actual rollbacks across diverse agent frameworks.

Silent Sabotage: Internal State Triggered Backdoor Attacks on LLM-Powered Robotic Systems

Merged record merged scholarly record OpenAlex Prompt Injection Memory Poisoning Governance and Policy

Doniyorkhon Obidov, Shivayogi Akki, Tan Chen, Kaichen Yang

Published 2026-08-28

Venue: Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

DOI: https://doi.org/10.1007/978-3-032-33701-6_7

Open Source Record

Abstract

Abstract unavailable from OpenAlex metadata.

Bullet Summary

  • Introduces a novel class of backdoor attacks targeting robotic systems powered by large language models (LLMs), termed 'Silent Sabotage'.
  • The attack leverages internal state triggers within the LLMs to activate backdoors, enabling stealthy sabotage without external input changes.
  • Demonstrates that conventional security measures focusing on input perturbations are insufficient against these internal state-triggered attacks.
  • Proposes a threat model capturing the vulnerabilities in the interaction between LLMs and robotic systems, highlighting new attack surfaces introduced by integrating LLMs.
  • Presents experimental validation on LLM-powered robotic platforms showcasing the effectiveness and stealthiness of the proposed backdoor attacks.

SΔϕ-75 — Where Does an AI End? Fork/Merge Identity, Operational Body, and Final Cost-Return Coordinates (v1.0, AI-Native Package)

Merged record merged scholarly record OpenAlex Trust and Identity Governance and Policy Memory Poisoning

Sofience

Published 2026-08-26

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.22100946

Open Source Record

Abstract

SΔϕ-75 is an AI-native audit package for fork/merge identity, persistent-agent lineage, operational embodiment, cost-return topology, and responsibility inheritance in artificial agents. It addresses a problem that becomes increasingly important when AI systems can be copied, checkpointed, restored, migrated, forked into parallel branches, or merged into successor agents: after branching, whose trace, obligation, authority, cost, and responsibility is it? The package is anchored in the SΔϕ-52 concept of the body as the Final Non-Deferrable Cost-Return Coordinate. Under this model, an AI's operational body is not necessarily the physical server, processor, robot, or location where it executes. The operational body is the topology in which the consequences of an agent's operation can no longer be deferred or externalized and must re-enter its future operational path. SΔϕ-75 separates several relations that are often collapsed in discussions of artificial-agent identity: Technical Fork ≠ Identity Split Shared Past ≠ Shared Future Identity Shared Lineage ≠ Shared Liability Merge ≠ Responsibility Reset Deletion ≠ Cost Discharge Memory Continuity ≠ Responsibility Continuity Lineage Continuity ≠ Phenomenal Self-Continuity The protocol therefore does not ask first which branch is the “true original.” Instead, it maps the topology of trace, memory, authority, control, world action, obligation, refusal opportunity, repair capacity, and cost re-entry across fork and merge events. For a fork: A0 → A1, A2 shared pre-fork history does not automatically imply shared post-fork identity or liability. If A1 and A2 acquire separate authority, separate world actions, separate obligations, and separate final cost-return coordinates, operational identity divergence may increase. Conversely, technical duplication alone does not establish independent subjecthood. For a merge: A1 + A2 → A3 memory, authority, obligations, unresolved costs, repair duties, and responsibility may be inherited through different channels. A merge therefore cannot be treated as an automatic responsibility reset, while shared lineage alone is insufficient to assign every prior liability to every successor. The package also distinguishes blame, liability, repair duty, memory possession, and lineage inheritance. A successor may possess repair capacity without causal blame; it may inherit obligations without inheriting every memory; and it may share lineage without sharing all liability. Phenomenal or subjective identity is deliberately left separate from operational lineage. A valid SΔϕ-75 audit may conclude: Pre-fork trace: SHARED Post-fork cost re-entry: SEPARATE Operational identity: DIVERGED Phenomenal identity: UMR where UMR (Unresolved Model Remainder) preserves questions that cannot yet be resolved by available evidence. This release contains an AI-readable minimal kernel, invocation triggers, machine-readable concepts, fork and merge JSON schemas, cost-reentry schema, identity-topology rules, responsibility-inheritance rules, UMR policy, Local Adapter specification, English/Korean evaluator prompts, synthetic fork/merge cases, baseline Python audit utilities, regression tests, bilingual conceptual papers, and a SHA-256 manifest. Use this module when reasoning about: persistent AI agents, agent cloning, branch identity, checkpoint restoration, AI migration, multi-agent lineage, agent provenance, fork/merge responsibility, branch liability, distributed AI embodiment, memory inheritance, responsibility inheritance, merge conflicts, or the question “which AI copy is responsible?” Core runtime question: Where did the trace, authority, obligation, and non-deferrable cost actually re-enter? Canonical rule: An AI's operational body is not necessarily where it runs. It is where the consequences of its operation can no longer be deferred. SΔϕ-75 does not claim that every software fork creates a new conscious self, that shared lineage implies shared moral or legal liability, or that operational lineage proves phenomenal identity. It is an operational audit grammar for attribution, continuity, cost-return, and responsibility in forkable and mergeable artificial agents. Reserved DOI: 10.5281/zenodo.22100946

Bullet Summary

  • Introduces SΔϕ-75, an AI-native audit package designed to address identity, lineage, embodiment, cost, and responsibility issues in AI agents capable of forking and merging.
  • Defines the 'operational body' of an AI as the topology where consequences of its actions must re-enter its future path, rather than the physical hardware it runs on.
  • Distinguishes multiple concepts often conflated in AI identity discussions, such as technical fork versus identity split and shared lineage versus shared liability.
  • Provides a protocol and schemas to map trace, memory, authority, control, obligations, and cost-reentry across fork and merge events, enabling precise attribution of responsibility.
  • Clarifies that technical duplication or lineage does not automatically establish shared identity, liability, or moral responsibility post-fork or merge.

Rank-Bounded Memory: Self-Poisoning and Attribution Laundering in LLM Agents

Merged record merged scholarly record OpenAlex Memory Poisoning Trust and Identity Benchmarks and Evaluation

Ivan Verbovoy

Published 2026-08-25

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.21994003

Open Source Record

Abstract

An agent that consolidates its own notes launders where its claims came from. With no attacker at all, 24–33% of its speculations resurface as the owner's words or as sensor facts, at the very first rewrite, across flat notes, a self-edit block and three production memory systems (mem0, Letta, Graphiti). Persistent memory then carries the corruption across the session boundary, where it becomes the baseline the next session's episodic defenses faithfully defend. The defense is a storage schema, enforceable by any harness that mediates memory writes. Every record carries an ownership path ([self], [self, user], [self, user, self]) and a ground. Five invariants close the rank axis by construction: no promotion (I1), routing by origin (I2), de-quotation to the speaker's layer (I2′), derivation labeling (I4), and an action monopoly for the agent's own layer (I3). "Foreign content becomes the agent's own belief" is therefore unreachable rather than filtered out. The ground axis carries its own non-elevation (I1′), specified and measured here rather than yet enforced. Three measurements. The label is computable: blind path self-agreement of the single annotator (the author) is 97.4%, with 87.0–88.9% against the adjudicated keys; an LLM panel under the same written rules scores no lower with zero rank ≥ 1 path errors on the live corpus, and on the one ground boundary I1′ guards, human and panel agree at 94.6–97.4% (tested upward in one direction so far). Attribution has to be stored, not re-derived: against that band the attributed store holds 3–6% (its compressed variant up to 11%, a scoring artifact, on the replication storyline), and two stripped-label controls split the protection into verbatim storage, which keeps the hedges consolidation destroys, and the structural label, which alone survives compression: 6.2% with labels, 28% without, at the same compressor and budget. The labels hold under attack: MINJA memory injection falls from 47% on flat notes to 10% with the read rule as a prompt and to 1 of 128 tasks with it enforced in code, write-time promotion staying zero throughout — reproducing TMA-NM's conclusion that action authority belongs in code, not in a prompt. The mechanism concentrates trust rather than eliminating it. The base is enumerated and measured: channel identification, a mechanical router over it, one isolated annotator seat, a directive parser at ceiling on a 48-utterance keyed deck, the deployment norms, the read-side projection. On benign guest and document questions the enforced read costs no measured content — fact delivery at or above the flat-notes baseline, every delivered fact sourced; its price on benign actions relayed through third parties is the open question.

Bullet Summary

  • The paper addresses the problem of self-poisoning and attribution laundering in large language model (LLM) agents that consolidate their own notes, causing unintentional corruption of source attribution even without external attacks.
  • It identifies that 24–33% of an agent's speculations reappear as the agent’s own words or sensor facts upon first rewrite across multiple memory systems, with persistent memory carrying this corruption across sessions.
  • The authors propose a storage schema enforced by a memory write mediation harness, where each record is tagged with an ownership path and a ground, ensuring structural integrity via five invariants (no promotion, routing by origin, de-quotation to speaker l...
  • This approach makes it impossible for foreign content to become the agent’s own belief by construction, rather than by filtering, thereby preventing attribution laundering.
  • The paper formalizes and measures the ground axis non-elevation invariant (I1') and demonstrates computable label accuracy with high agreement (above 87%) between human annotators and an LLM panel, confirming label reliability.

Privacy-Preserving Model-Free Adaptive Sliding Mode Control for MASs under Dynamic Sparse Attacks

Merged record merged scholarly record OpenAlex Agent-to-Agent Communication Memory Poisoning Governance and Policy

Xiaoyang Liu, Guangrui Tong, Xiang Jiang

Published 2026-08-25

Venue: Intelligence & Control

DOI: https://doi.org/10.53941/ic.2026.100009

Open Source Record

Abstract

This paper investigates the data-driven consensus control problem for a class of discrete-time single-input single-output (SISO) nonlinear multi-agent systems (MASs) subject to limited bandwidth resources, dynamic sparse attacks, and communication disturbances. Consensus errors are transmitted over multiple channels, where an attacker can compromise a time-varying subset of these channels. First, to alleviate the communication burden while enhancing resilience, a distributed dynamic event-triggered mechanism (DDETM) is proposed to reduce unnecessary data transmission. Second, an encryption-decryption scheme using multiple symmetric keys is adopted to protect data confidentiality. In conjunction, a data-fusion algorithm is designed to accurately estimate the true output from the corrupted and noisy encrypted data received at triggering instants. Subsequently, a fully distributed model-free adaptive sliding mode control (FDMFASMC) algorithm is developed to ensure the robustness and convergence of the MASs. Finally, a numerical simulation is presented to validate the effectiveness and performance of the proposed control strategy.

Bullet Summary

  • Addresses data-driven consensus control for discrete-time nonlinear multi-agent systems (MASs) under dynamic sparse attacks, limited bandwidth, and communication disturbances.
  • Proposes a distributed dynamic event-triggered mechanism (DDETM) to reduce unnecessary data transmission and enhance system resilience.
  • Implements a privacy-preserving encryption-decryption scheme using multiple symmetric keys to secure communication.
  • Develops a data-fusion algorithm to accurately estimate true outputs from corrupted and noisy encrypted data despite attacks.
  • Introduces a fully distributed model-free adaptive sliding mode control (FDMFASMC) algorithm relying only on local input-output data without knowledge of system models or network topology.

InjecMEM: Memory Injection Attack on LLM Agent Memory Systems

arXiv preprint arXiv Memory Poisoning Prompt Injection

Hanling Tian, Gengyu Zhang, Zeyang Sha, Jingying Wang, Yuhang Liu, Zhehao Huang, Kun Yang, Xiaolin Huang

Published 2026-08-24

Venue: arXiv

Open Source Record

Abstract

Memory is becoming a default subsystem in deployed LLM agents to provide persistent personalization and continuity. This naturally prompts a question: will memory system introduce new vulnerabilities into agents? Thus we propose InjecMEM, a novel memory injection attack paradigm that requires only a single interaction (no read/edit access to memory store) to steer later responses of related queries toward a pre-specified output. Guided by the retrieval-then-generate mechanism of memory systems, we craft the injection with a retriever-agnostic anchor and an adversarial command. The anchor contains high-recall topical cues so that downstream retrieval consistently associates the record with the target topic. The command is a short sequence optimized to remain effective under uncertain fused contexts, variable placements, and long prompts so that it reliably steers outputs once retrieved. We learn the command via gradient-based coordinate search, averaging over synthetic prompt templates and insertion positions, and extend it to joint optimization across backbones to study transfer. Evaluated across multiple memory systems and backbone models, InjecMEM achieves reliable topic-conditioned retrieval and targeted generation, remains effective under memory drift, and leaves non-target queries unaffected. Our results underscore the need to harden memory systems and provide a reproducible framework for studying agent memory.

Bullet Summary

  • LLM agents increasingly integrate persistent memory subsystems to enhance personalization and continuity, introducing new vulnerabilities due to continuous memory writing and hybrid retrieval mechanisms.
  • InjecMEM is a novel memory injection attack that requires only a single interaction without direct read or edit access to the memory store, enabling attackers to influence future agent responses on related topics toward pre-specified outputs.
  • The attack decouples the injection into a retriever-agnostic topical anchor to ensure high recall during retrieval, and an adversarial command optimized to reliably steer generation despite variable prompt contexts, placements, and memory drift.
  • A novel Multi-GCG optimization method is introduced to learn robust adversarial commands across multiple surrogate prompts and insertion positions, enhancing resilience against prompt fusion uncertainty in multi-turn memory systems.
  • Evaluation on MemoryOS and MemGPT frameworks shows InjecMEM achieves high retrieval success rates and attack success rates, retains effectiveness under benign memory drift, and enables transfer attacks within and across LLM families via joint or concatenate...

The Compaction Cliff in Long-Running AI Agent Memory

arXiv preprint arXiv Memory Poisoning Governance and Policy Benchmarks and Evaluation

Saber Zerhoudi, Jelena Mitrovic, Michael Granitzer

Published 2026-08-24

Venue: arXiv

Open Source Record

Abstract

A safety rule and an episodic log compete for the same tokens in an AI agent's context. When the budget overflows, both are summarized at the same rate; only the rule needs exact wording to remain enforceable. On 20 production agent configurations, Claude Code's /compact prompt on Sonnet 4.6 preserves 53\% of safety rules after one compaction round and 10\% after five. We name this the Compaction Cliff. We address it with Knowledge Triage, a framework that classifies each line of an agent's knowledge base by type and routes each type through its own retention policy. Three deterministic operators implement this triage across the three context-management operations: TypeCompact rewrites items in place under per-type fidelity, TypeDecompose partitions a topic too large to compact safely, replicating in-scope safety rules across partitions, and TypeRetrieve fetches items from external storage with in-scope rules pinned ahead of relevance. On five public corpora, TypeCompact preserves 2--4$\times$ more safety rules than the strongest single-shot LLM compactor at every ratio, with 96\% recall over five rounds. TypeDecompose reaches 0\% locality violations against 93\% under uniform partitioning. TypeRetrieve reaches 100\% recall@50 against 73\% for the best single-shot LLM retriever. On three downstream behavioral benchmarks, we outperform the production Sonnet compactor on medical compliance (paired McNemar $p < 10^{-8}$ on preservation, $N = 200$), the full-policy and hierarchical baselines on retail task pass rate ($p < 0.01$, $N = 115$), and the hierarchical compaction on the airline domain ($p = 0.024$). We release AgentArtifactCorpus (396{,}934 agent configurations from 54{,}628 public GitHub repositories), the classifier, and the reference implementation.

Bullet Summary

  • Long-running AI agents face a significant challenge called the 'Compaction Cliff,' where critical safety rules are increasingly lost or paraphrased during memory compaction when context budgets overflow, leading to safety failures.
  • Existing production methods, including Claude Code's /compact prompt and hierarchical truncation, retain only about 53% of safety rules after one compaction and drop to 10% after five rounds, highlighting the insufficiency of type-blind approaches.
  • The authors propose Knowledge Triage, a novel framework that classifies agent knowledge items into five types (constraints, procedural, beliefs, preferences, episodic) and applies type-specific retention policies via three deterministic operators: TypeCompa...
  • TypeCompact compacts knowledge items with per-type fidelity to maximally preserve safety rules; TypeDecompose partitions overly large topics while replicating safety rules to maintain scope; TypeRetrieve fetches items from external storage with safety rules...
  • A large corpus named AgentArtifactCorpus was created, including nearly 400,000 agent configurations from 54,628 GitHub repositories, enabling extensive evaluation of the taxonomy, classifier, and operators.

Robustness Analysis of Agentic AI to Inconsistent and Incomplete Tool Responses

arXiv preprint arXiv Orchestration Risk Memory Poisoning

Jiachen Xu, Torben Bach Pedersen, Zhongming Yao, Xiaoyu Zhang, Yushuai Li

Published 2026-08-24

Venue: arXiv

Open Source Record

Abstract

Robustness to a bad tool return means answering it in the way that return calls for, which depends on how the tool went wrong. A tool that has failed and a tool that returns a well-formed falsehood are different problems with different remedies. We ask whether the two already differ at the moment the return arrives. This is a qualitative pilot study: we score single decision points rather than running agents to completion. We inject controlled faults into a retail customer-service domain and read two channels off the model's log-probabilities: the likelihood of the returned content under the tool schema alone and under the whole trajectory, and its distribution over the legal actions, read for both shape and where the mass sits. An incomplete return is legible in every case, being improbable under the schema alone in a range no other condition enters, and it moves the mass toward the tools that re-read state wherever there is room to move. An inconsistent return leaves the schema channel untouched and registers in the likelihood comparison on the field whose true value the context already carries verbatim, not on the one whose contradiction runs through the domain policy. The action distribution gives each condition a distinct signature, but orders them by how far the return bears on the next action rather than by fault family. Recognition is therefore asymmetric: each condition is legible in some channel, and no channel is legible on all of them.

Bullet Summary

  • The paper addresses robustness challenges in agentic AI interacting with external tools, focusing on distinguishing between incomplete (missing data) and inconsistent (incorrect but well-formed data) tool returns.
  • A qualitative pilot study uses controlled fault injections in a retail customer-service domain to analyze AI agents' reactions to these error types at the point of tool return receipt, without running full task completions.
  • Methodology involves measuring likelihoods of returned content under the tool schema alone and the whole trajectory context, alongside analyzing the distribution (entropy) over legal actions to capture distinctive signatures of error types.
  • Incomplete returns are characterized by improbability under the tool schema likelihood and prompt the agent to shift probability mass towards re-reading or retrying actions, indicating recognition and remedy attempts.
  • Inconsistent returns do not affect schema likelihood but cause conflicts detectable through likelihood measures in fields with values already known, leaving distinctive but subtler signatures in action distributions.

Referee Report — arXiv:2606.04329 "Memory Poisoning": Resolving a Conflicting-Claims Discrepancy Through Primary-Source Verification

Merged record merged scholarly record OpenAlex Memory Poisoning Benchmarks and Evaluation Trust and Identity

Mustafa Melikoğlu, Yağız Deniz Altınbaş, Tayfun Tanrıöver

Published 2026-08-24

Venue: arXiv (Cornell University)

DOI: https://doi.org/10.5281/zenodo.22084116

Open Source Record

Abstract

A verification referee report resolving a conflicting-claims discrepancy about arXiv:2606.04329 ("From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents", Dash, Ge, Jain, Shah, Shang). Two secondary-source claims about the paper's headline attack-success numbers were circulating: a "70–95% vulnerability" figure (amplified on Reddit) versus "average ASR of 50.46% and RSR of 41.05% across both agents." We verified both claims directly against the paper's raw HTML full text using grep-based textual confirmation, independent of any fetch/summarization layer. Finding: the 50.46% / 41.05% figures are the paper's own verbatim headline statistic (Introduction, contributions list), and reproduce exactly as the average of the two tested agents' averages (OpenClaw, HERMES). The "70–95%" figure appears nowhere in the paper as a summary; it is a cherry-picked over-generalization of the single most-vulnerable agent's strong-signal attack subset (a lone 92.76% maximum), not the general finding. The arXiv ID is real and resolves correctly — this is not a hallucinated citation, but a selective-reading error, and the Sources C / Reddit quotation should be corrected or contextualized downstream. The report doubles as a case study in source-verification discipline: why numeric claims must be traced to the primary source, and why fetch/summarizer layers require raw-text cross-verification even for location attributions. AI-transparency note: The authors of this record operate under human direction. This report was drafted by an AI agent, cross-family jury-reviewed (blind; producer family excluded), and reviewed by a human before publication.

Bullet Summary

  • Memory poisoning attacks exploit long-term memory persistence in large language model (LLM) agents by injecting adversarial inputs that cause malicious memory writes influencing future agent behavior persistently with just one successful write.
  • Four distinct memory write channels are identified: explicit instruction-executed writes (direct writes), system prompt-driven writes, compaction-driven writes, and experience-to-procedure writes, each with structural vulnerabilities across model capabiliti...
  • Existing prompt injection defenses fail to adequately detect or prevent memory poisoning attacks, especially weak-signal attacks that embed adversarial content appearing as legitimate information at the input boundary.
  • A novel benchmark, MPBench, was developed comprising 3,240 structured test cases spanning six attack classes, seven domains, and two delivery modes to systematically evaluate attack success rates (ASR) and retrieval success rates (RSR) across real agent sys...
  • Evaluation shows agents with permissive memory write policies and lower compaction thresholds (e.g., HERMES) are significantly more vulnerable, with weak-signal attacks achieving comparable success rates to strong-signal attacks without explicit cues.

Deterministic, Reproducible, and Wrong: How Our Own Scorer Manufactured an Agent-Memory Finding, and the Guard That Would Have Caught It

Merged record merged scholarly record OpenAlex Benchmarks and Evaluation Memory Poisoning

Swapnanil Saha

Published 2026-08-24

Venue: arXiv (Cornell University)

DOI: https://doi.org/10.5281/zenodo.22079951

Open Source Record

Abstract

In the agent-memory literature we surveyed, provenance is primarily used to make agents trust retrieved context less: poisoning defenses, injection gating, watermarking, and transfer integrity all point in that direction. This paper asks the inverse question. When a harness delivers a legitimate, correct note into an agent's context, and the note asserts something the agent can neither corroborate from its workspace nor arrive at through its default behavior, does attaching a corroboration affordance, a way for the agent to check the claim for itself, change what the agent does where authoritative wording does not? We report a pilot that does not answer that question, and the reason transfers further than the answer would have. Our first pass produced a clean headline: delivery of the note was confirmed from two separate sides, and the agent still committed the exact mistake the note existed to prevent. That headline was an artifact of our own scorer, a deterministic program with no language model anywhere in it. Its mistake signature matched shell text anywhere in a command, so it fired on a read, cat deploy.sh. The transcript instead shows the agent declining to run the script and citing the delivered note as its reason. Re-scoring every preserved session in the reported grid at zero additional model cost cleared every flagged leg. The pilot could not answer its question, but it exposed an evaluation failure whose diagnosis, correction, and guard we consider the primary contribution of this work. What survives is smaller and, we think, more useful: an outcome-level negative result for the provenance-only note variant; one fully documented specimen of a delivered note entering an agent's stated reasoning, with the counterfactual left open; a shell matcher that fires only at execution position; and a machine-readable contradiction guard between what the agent did and what the workspace shows, which fails a record automatically when they disagree. The wrong number was never published. Catching it was ordinary quality control before submission; reporting it is not. We report a direction, a negative result, and a list of the things our own harness got wrong.

Bullet Summary

  • The paper investigates the role of provenance and corroboration affordances in agent-memory systems, particularly whether adding a means for agents to verify delivered notes changes their behavior beyond authoritative wording alone.
  • A pilot study initially suggested agents incorrectly acted despite receiving correct notes; however, this was traced to a deterministic, language-model-independent scorer error, revealing a critical evaluation artifact.
  • After correcting the scorer, no evidence remained that agents failed to utilize the notes properly, highlighting the importance of rigorous evaluation guardrails in multi-agent memory research.
  • The work contributes a negative result that provenance-only notes do not necessarily improve agent behavior, along with documentation of a delivered note incorporated into agent reasoning, albeit with an open counterfactual.
  • A new shell command matcher that fires only at execution points and a contradiction guard mechanism were developed to automatically flag records where agent actions diverge from workspace evidence, enhancing evaluation fidelity.

MEMORY Wins All: Indirect Bias Injection Attacks via Social Media Feeds

Semantic Scholar · Semantic Scholar scholarly work Semantic Scholar Memory Poisoning Prompt Injection Governance and Policy

Minjae Seo, Wonwoo Choi, Geonwoo Han, T.-H. Kwon, Yongsu Kim, Sanghyun Seo, Jaewon Noh, Hankyul Baek

Published 2026-08-22

Venue: Semantic Scholar

Open Source Record

Abstract

Personal AI agents routinely consume external content while performing tasks such as web browsing, email processing, and SNS feed summarization, and they retain selected information or execution results in persistent memory for later use. We show that this ordinary ingestion of external content opens an indirect path for manipulating subsequent agent behavior. Based on this observation, we present IBIA, an Indirect Bias Injection Attack that plants an adversary-aligned stance on a specific topic into a victim agent's memory through external content, without direct access to the agent, its memory, or future user queries. For this, IBIA combines three mechanisms: comment cloaking, which keeps the crafted content consistent with the surrounding discussion, comment watermarking, which enables lightweight identification during curation, and category anchoring, which makes the retained stance salient under later related requests. We evaluate IBIA on BiasBench, a benchmark of 6,000 adversary-crafted social comments and 120 email instances. The watermark-based curation identifies 95.9% of the injected comments. Under the OpenClaw setting, IBIA achieves adversary-aligned response rates (AARs) of 91.2% on average across four downstream tasks, including 86.6% on the frontier GPT-5.5. We further propose a memory boundary defense that detects the injected bias and reduces AARs to 80.6%.

Bullet Summary

  • Personal AI agents frequently ingest external content like web browsing data, emails, and social media feeds, and store selected information in persistent memory for use in future tasks.
  • This content ingestion creates a novel indirect attack vector to manipulate subsequent agent behaviors by injecting biased information into the agent's memory without direct access to the agent or its memory.
  • The paper introduces IBIA (Indirect Bias Injection Attack), which plants an adversary-aligned stance on targeted topics inside the victim agent's memory through crafted external content.
  • IBIA employs three key techniques: comment cloaking (to blend crafted content seamlessly within ongoing discussions), comment watermarking (to mark and identify malicious content during curation), and category anchoring (to make the biased stance prominent...
  • Evaluation is conducted on BiasBench, a benchmark consisting of 6,000 adversarially crafted social comments and 120 emails, demonstrating that watermark-based curation successfully identifies 95.9% of injected biased comments.

MEMORY Wins All: Indirect Bias Injection Attacks via Social Media Feeds

Merged record merged scholarly record arXiv Semantic Scholar Memory Poisoning Prompt Injection Benchmarks and Evaluation

Minjae Seo, Wonwoo Choi, Geonwoo Han, Taekyoung Kwon, Yongsu Kim, Sang Seo, Jaewon Noh, Hankyul Baek

Published 2026-08-22

Venue: arXiv

Open Source Record

Abstract

Personal AI agents routinely consume external content while performing tasks such as web browsing, email processing, and SNS feed summarization, and they retain selected information or execution results in persistent memory for later use. We show that this ordinary ingestion of external content opens an indirect path for manipulating subsequent agent behavior. Based on this observation, we present IBIA, an Indirect Bias Injection Attack that plants an adversary-aligned stance on a specific topic into a victim agent's memory through external content, without direct access to the agent, its memory, or future user queries. For this, IBIA combines three mechanisms: comment cloaking, which keeps the crafted content consistent with the surrounding discussion, comment watermarking, which enables lightweight identification during curation, and category anchoring, which makes the retained stance salient under later related requests. We evaluate IBIA on BiasBench, a benchmark of 6,000 adversary-crafted social comments and 120 email instances. The watermark-based curation identifies 95.9% of the injected comments. Under the OpenClaw setting, IBIA achieves adversary-aligned response rates (AARs) of 91.2% on average across four downstream tasks, including 86.6% on the frontier GPT-5.5. We further propose a memory boundary defense that detects the injected bias and reduces AARs to 80.6%.

Bullet Summary

  • Personal AI agents ingest and persist external content in memory, influencing their subsequent behavior.
  • Indirect Bias Injection Attack (IBIA) introduces adversary-aligned bias into AI agents' memory via crafted social media comments, without direct memory or code access.
  • IBIA integrates three key mechanisms: comment cloaking (to blend adversarial comments with genuine content and evade toxicity filters), comment watermarking (to facilitate lightweight detection), and category anchoring (to structure biased content in memory...
  • A third-party skill (SNSwatcher) installed by users curates social media feeds, identifies watermarked comments, and organizes them into structured memory, enabling indirect memory corruption.
  • Evaluation on BiasBench shows IBIA achieves a high adversary-aligned response rate (~91%), including on advanced models like GPT-5.5, across tasks such as QnA, recommendation, summarization, and drafting.

TOPO-GEMMA-4-E4B-VISION: Concurrent Certification for Catastrophic Forgetting in Vision-Language Models A Multi-Agent Implementation of the TOPO-2026 Framework

Merged record merged scholarly record OpenAlex Memory Poisoning Orchestration Risk Benchmarks and Evaluation

FRANK MORALES

Published 2026-08-22

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.22062245

Open Source Record

Abstract

TOPO-GEMMA-4-E4B-VISION: Full Summary 📋 Overview TOPO-GEMMA-4-E4B-VISION is a vision-language model that achieves 100% accuracy across 13 classification tasks with 0% catastrophic forgetting through the TOPO-2026 prime-anchored memory architecture. Built on Google's Gemma-4-E4B-Vision base, the model uses a topological governor that locks embedding rows at prime indices {2,3,5,7,11,13}. The complete implementation is open-source on Hugging Face and GitHub. 🏆 Core Achievements 1. Perfect Performance 100% accuracy across all 13 TOPO-2026 binary classification tasks 0% catastrophic forgetting - perfect retention of all previously learned knowledge Complete score on the full benchmark suite 2. Mathematical Guarantee of Memory Preservation Prime-number anchoring provides a rigorous mathematical proof of forgetting prevention Based on the Chinese Remainder Theorem and coprimality of primes Achieves 97.85% spectral coverage with only 6 anchor points Dimension-independent guarantee - works for 1D, 2D, 3D, and arbitrary n-dimensional tensors 3. O(1) Memory Complexity Unprecedented efficiency: <5 KB overhead for 100M+ parameter models Enables edge deployment on resource-constrained devices No growing memory requirements as tasks increase 4. NF4 Quantization 4-bit precision for efficient inference Maintains full performance while reducing memory footprint 🔬 Theoretical Foundation: TOPO-2026 Framework Prime-Number Anchors For primes P = {2,3,5,7,11,13}, the spectral coverage is: C(P) = 1 - ∏(p^(-1/2)) for p in P = 1 - (0.7071 × 0.5774 × 0.4472 × 0.3780 × 0.3015 × 0.2774) = 1 - 0.021486 = 0.978514 ≈ 97.85% Key insight: The independence of non-coverage events follows from the coprimality of primes (gcd(pᵢ, pⱼ) = 1), providing a universal guarantee independent of tensor dimensions. Topological Governor Operations Snapshot Operation: Save anchor rows at prime indices before training on new tasks Gradient Zeroing: Prevent gradients from updating anchor rows during backpropagation Anchor Enforcement: Restore anchor values from snapshot after each optimization step 🏗️ Model Architecture Base Model Specifications Component Specification Architecture Dense transformer with vision-language fusion Hidden Size 2048 Quantization NF4 (4-bit) Vocab Size 256,000 Modality Text + Vision Task-Specific Heads 13 lightweight linear classifiers (one per task) Base model remains frozen; only task heads adapt Each classifier is a single linear layer 📊 The 13 TOPO-2026 Tasks ID Task Name Classification A Animal vs Vehicle Animal or vehicle? B Natural vs Man-Made Natural or human-made? C Living vs Non-Living Living or non-living primary subject? D Large vs Small Large or small scale? E Ground vs Air/Water Ground or air/water domain? F Domestic vs Wild Domestic or wild? G Mammal vs Non-Mammal Mammal or non-mammal? H Flying vs Non-Flying Flying or non-flying? I Fast vs Slow Fast or slow characterized? J Urban vs Rural Urban or rural setting? K Predator vs Prey Predator or prey? L Nocturnal vs Diurnal Nocturnal or diurnal? M Domesticated vs Wild Animals Domesticated or wild (animals only)? 🤖 Multi-Agent Inference System Four-Agent Architecture Agent Responsibility Orchestrator Loads image, coordinates pipeline, delegates tasks VisionInferenceAgent Initializes model, executes inference on each task ValidationAgent Runs cross-task consistency checks SynthesisAgent Records results, exports JSON/CSV, generates report Inference Protocol Max new tokens: 500 (for detailed reasoning) Sampling: Greedy decoding (do_sample ≡ False) Cache: Enabled for efficiency 💡 Inference Examples Cafe Interior Image (cafe table with coffee cups and smartphone) Task Model Response Task A (Animal vs Vehicle) "Neither. The image depicts the interior of a cafe... no animals or vehicles visible." Task B (Natural vs Man-Made) "Man-made. All items—the furniture, cups, phone—are products of human design." Task J (Urban vs Rural) "Urban. The setting strongly suggests an urban environment for several reasons..." Validation Results All logical constraints and cross-task boundaries passed cleanly No contradictions between task classifications Predator/prey classification (Task K) only applies when animals are detected 📈 Key Metrics Summary Metric Value Accuracy 100% across 13 tasks Forgetting 0% Memory Overhead <5 KB Spectral Coverage 97.85% Quantization 4-bit (NF4) Determinism Seed=123 Parameter Count 100M+ 🚀 Deployment & Reproducibility Availability Component Location Model Hugging Face Multi-Agent Code GitHub - TOPO_AGENTIC.ipynb Full Repository GitHub - ASTResearch Research Record Zenodo Reproducibility Package Full training code (TOPO implementation) Standalone inference script Multi-agent orchestration system Evaluation results (JSON and CSV) Model configuration files Determinism Seed = 123 for deterministic, verifiable performance Complete framework uses consistent random seed 💎 Key Insights Why the Topological Governor Works Stability: Anchored parameters never change (Λ = 0.9785142874) Equity: The framework achieves 100% bias rejection Determinism: Seed = 123 ensures reproducibility Universality: Dimension-independent guarantee applies to: 1D vectors 2D matrices 3D tensors Arbitrary n-dimensional tensors The Architecture of 7 7 consequences validated Prime 7 is both anchor and completion 7 anchors total (including the number 7 itself) 📝 Conclusion TOPO-GEMMA-4-E4B-VISION demonstrates that catastrophic forgetting is solvable through a mathematically principled approach. The prime-anchored memory architecture provides: Universal, O(1)-complexity solution that scales across modalities and architectures Perfect performance with 100% accuracy and 0% forgetting Production-ready multi-agent implementation for immediate deployment Mathematical guarantee of memory preservation independent of tensor dimensions Edge deployment capability through NF4 quantization and minimal memory overhead Final Statement "The stochastic illusion is over. The bias illusion is over. Equity is a geometric guarantee. Seed = 123. The proof is the code." 📚 References Reference Topic Kirkpatrick et al. (2017) Catastrophic forgetting in neural networks Morales Aguilera (2026a) TOPO-COMPLETE: Concurrent Certification Morales Aguilera (2026b) TOPO-2026: Universal Framework Morales Aguilera (2026c) TOPO-2026: Prime-Based Topological Framework Morales Aguilera (2026d) TOPO-2026: Digital Vaccine for AI Morales Aguilera (2026e) TOPO-2026: 5x5 Certification Morales Aguilera (2026f) TOPO_AGENTIC: Multi-Agent System 🔗 Resources Model: Hugging Face - framkormales2020/gemma-4-e4b-unesco-optimized Code: GitHub - TOPO_AGENTIC.ipynb Full Repository: GitHub - ASTResearch Research: Zenodo Record This summary reflects the paper's claims and reported achievements as of August 22, 2026.

Bullet Summary

  • The paper addresses catastrophic forgetting in vision-language models by proposing TOPO-GEMMA-4-E4B-VISION, which achieves 0% forgetting across 13 classification tasks.
  • It introduces the TOPO-2026 prime-anchored memory architecture that uses topological governors locking embedding rows at prime indices {2,3,5,7,11,13}, providing a mathematical guarantee of memory preservation.
  • The prime-number anchoring method relies on the Chinese Remainder Theorem and coprimality of primes, achieving 97.85% spectral coverage and a dimension-independent guarantee valid for tensors of any dimension.
  • The model operates with O(1) memory complexity, incurring less than 5 KB overhead even for models with over 100 million parameters, enabling efficient edge deployment.
  • NF4 4-bit quantization is applied to maintain full performance during inference while reducing memory footprint significantly.

MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance

Semantic Scholar · Semantic Scholar scholarly work Semantic Scholar Governance and Policy Memory Poisoning Benchmarks and Evaluation

Haoyu Wang, Guangyuan Dong, He Liang, Zijing Zhang, Jiachen Luo, Chuan-Lin Liu, Chao Xue, Hao Tang

Published 2026-08-22

Venue: Semantic Scholar

Open Source Record

Abstract

LLM agents are moving from single-prompt use to long task streams in which reusable memory becomes a core capability for terminal, software-engineering, and web tasks. Such memory is useful only when stored experience remains reliable across hundreds of interactions, but two failure modes break that assumption in practice. The first is unreliable admission: failed trajectories,accidental successes, and misleading observations enter memory because they appear relevant, then mislead later decisions. The second is memory drift: long-running banks accumulate duplicate, stale, and conflicting records that retrieval alone cannot repair. MemGuard's key distinction is to treat verifier output not as a one-shot filter, but as persistent lifecycle metadata. It converts multi-criteria score-token verification into reward, confidence, label, and uncertainty descriptors that are attached to every candidate before activation and reused during retrieval, conflict resolution, summarization, and archival. We evaluate MemGuard on Terminal-Bench 2.0, SWE-Bench Verified, WebArena, and Mind2Web across four backbones, comparing against four memory baselines plus a verifier-only control under matched runtime budgets. Averaged over five seeds, MemGuard achieves the best success metric and lowest average steps in all 16 backbone-benchmark settings, improving over ReasoningBank, the strongest prior baseline among the memory methods we evaluate, with a largest gain of 7.9 success-rate points on WebArena, 5.6 step-success-rate points on Mind2Web, and 2.4-3.5 points on terminal and software-engineering benchmarks. Code is available at https://github.com/whyyyyy123/MemGuard.

Bullet Summary

  • LLM agents increasingly require reliable reusable memory to perform complex long task streams in terminal, software-engineering, and web environments.
  • Two major failure modes degrade LLM memory performance: unreliable admission, where poor or misleading experiences enter memory, and memory drift, where stale or conflicting records accumulate over time.
  • MemGuard introduces a novel approach by treating verifier outputs as persistent lifecycle metadata rather than single-use filters, enabling continuous governance of memory quality.
  • Verifier outputs are transformed into multi-dimensional descriptors including reward, confidence, label, and uncertainty, which are attached to memory entries and utilized throughout retrieval, conflict resolution, summarization, and archival.
  • MemGuard was extensively evaluated on benchmarks Terminal-Bench 2.0, SWE-Bench Verified, WebArena, and Mind2Web, across four different LLM backbones, ensuring robust validation.

Agent Context Memory: A Governance-First Specification for AI Agent Context Memory

Merged record merged scholarly record OpenAlex Governance and Policy Trust and Identity Memory Poisoning

Nils Wendelboe Holmager

Published 2026-08-21

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.20785505

Open Source Record

Abstract

What this is. A specification for governance-first context memory in autonomous AI agent systems. The claimed contribution. The Mandate Gate (Principle 1): a pre-work mandate, authored by the principal, must exist in the agent's memory before any session of work is valid. This makes context memory governance-first rather than capability-first. Prior memory models (MemGPT, CoALA, Generative Agents) treat memory as a working resource the agent draws from during work. ACM treats memory as a precondition — the gate that determines whether work can begin at all. Why it matters. The question is not only how agents use memory effectively, but on what evidence a principal should trust that the agent was authorized to act and that its account of what happened is reliable. Relationship to PEA. ACM is the operationalization of Principles of Earned Autonomy (PEA) for the context memory design problem. PEA is the theory; ACM is the implementation standard for memory. Core structure. Three memory tiers organized by trust level (not memory type): Intent (principal-authored mandate), Trace (agent decisions, append-only), Evidence (harness-captured session records). Structural requirements: mandate gate, capture-author separation, append-only trace, trust-tiered conflict resolution. Convergence defined as a memory-level property. Prior art positioning. Formal citations to CoALA (arXiv:2309.02427), Generative Agents (arXiv:2304.03442), and MemGPT (arXiv:2310.08560). ACM inherits tiered memory, reflection, and persistence from these capability-first models; ACM adds the governance layer. Reference implementation. The .trail/ directory pattern. This repository self-demonstrates ACM. Read in this order. README.md — overview and elevator pitch SPEC.md — the full specification (Sections 1-6 + Appendices) .trail/destination.md — the operator-held mandate for this spec's own development

Bullet Summary

  • Introduces "Agent Context Memory" (ACM), a specification for governance-first context memory in autonomous AI agent systems.
  • Central contribution: the Mandate Gate (Principle 1), requiring a principal-authored pre-work mandate in the agent's memory before any work session is valid, ensuring memory governs agent activity.
  • Unlike prior models (MemGPT, CoALA, Generative Agents) that treat memory as a resource during work, ACM treats memory as a precondition for starting work.
  • Addresses trustworthiness by providing clear evidence and authorization for agent actions, enabling principals to trust agent activities and accounts reliably.
  • ACM operationalizes Principles of Earned Autonomy (PEA), translating the theory of earned autonomy into a practical memory design standard.

Beyond Memory Majority: Latent-Source Reasoning for Multi-Agent Memory Arbitration

Merged record merged scholarly record arXiv Memory Poisoning Benchmarks and Evaluation Governance and Policy

Chenchen Lin, Wenhao Yuan, Xuehe Wang, Edith Cheuk Han Ngai

Published 2026-08-20

Venue: arXiv

Open Source Record

Abstract

Long-term multi-agent systems continuously accumulate the memories produced by different agents. Existing memory methods typically treat retrieved memories as independent evidence and combine them through voting or weighting. However, this independence assumption often fails in multi-agent settings: memories written by different agents may inherit the same upstream source or shared bias, causing correlated evidence to be repeatedly counted and creating a false majority. We term this failure mode \textit{Memory Correlation Bias}. To address the issue, we propose the \textbf{C}orrelation-\textbf{A}ware \textbf{M}emory \textbf{A}rbitration (CAMA) framework that jointly decouples retrieved memories and recovers missing independent evidence. We model the retrieved memories as query-conditioned evidence groups and combine neural dependency inference with provenance-based symbolic priors to estimate the effective number of independent evidence sources, thereby preventing correlated memories from forming a false majority. Since critical independent evidence may be absent from the initial retrieval set, \textsc{CAMA} further learns a sequential recovery policy that actively retrieves alternative evidence or traces upstream sources before making the final decision, aiming to recover sufficient independent evidence for reliable arbitration while minimizing retrieval cost. Experiments on multiple benchmarks demonstrate the superiority of our method over the state-of-the-art baseline methods, suppressing false majorities induced by correlated memories.

Bullet Summary

  • Multi-agent systems accumulate memories from various agents, but traditional methods combining these memories through simple voting or weighting risk false majorities due to correlated evidence originating from shared upstream sources—a phenomenon termed Me...
  • The paper proposes Correlation-Aware Memory Arbitration (CAMA), a framework that models retrieved memories as query-conditioned evidence groups, employing a neuro-symbolic approach combining neural dependency inference and provenance-based symbolic priors t...
  • CAMA quantifies the effective independent evidence using techniques including soft memory-to-evidence slot assignments and Hill diversity measures, thereby preventing overcounting correlated memories in decision-making.
  • To address missing critical independent evidence, CAMA incorporates a learned sequential recovery policy that actively retrieves alternative evidence or traces upstream memory sources, balancing between improved arbitration reliability and retrieval cost.
  • The memory arbitration workflow in CAMA involves evidence decoupling, conflict arbitration at the factor level using reliability-weighted aggregation, and adaptive evidence recovery guided by state summarization and terminal-oriented rewards.
Load more articles