Memory Poisoning

Abstract

Autonomous agents driven by Large Language Models (LLMs) operating in extended multi-session settings suffer from two systemic failure modes: (1) recursive hallucination compounding, wherein unverified model-originated narration is committed to long-term memory and subsequently retrieved as established truth; and (2) fabricated action claims, wherein agents assert execution of irreversible real-world side effects without verified tool execution. We present the architecture and mechanical evaluation of Project Lucca, an open-source (GNU AGPLv3), local-first neuro-symbolic agent architecture designed for strict epistemic grounding. Lucca enforces two primary defenses: (1) a Six-Signer Cryptographic Provenance Gate that requires domain-separated HKDF HMAC-SHA256 signatures for persistent memory commits, enforcing an epistemic invariant at the database schema level that separates conversational continuity from factual grounding eligibility; and (2) a Deontic Truth-in-Receipt Output Supervisor that screens generated speech tokens against deterministic execution receipts before user presentation. We evaluate these mechanisms across an authored 73-test deterministic unit battery, a 100-probe live action-hallucination testbed across edge models (Gemma 4 E2B/E4B), and analyze documented production failure modes under compute constraints.

Summary

• Identifies two systemic failure modes in LLM-driven autonomous agents: recursive hallucination compounding and fabricated action claims without verified tool execution.
• Presents Project Lucca, an open-source local-first neuro-symbolic agent architecture focused on strict epistemic grounding to mitigate these failure modes.
• Introduces a Six-Signer Cryptographic Provenance Gate using domain-separated HKDF HMAC-SHA256 signatures to enforce epistemic invariants at the database schema level, separating conversational continuity from factual grounding eligibility.
• Develops a Deontic Truth-in-Receipt Output Supervisor that validates generated speech tokens against deterministic execution receipts before presenting them to users.
• Validates defenses with a comprehensive evaluation consisting of a 73-test deterministic unit battery and a 100-probe live action-hallucination testbed on edge models like Gemma 4 E2B/E4B.

Abstract

Decentralized multi-agent systems (MAS) enable large language model agents to interact without centralized control. Agents need to adapt to other agents at test time and reuse accumulated experience to refine their interaction strategies for future interactions across cooperative, mixed-motive, and adversarial settings. However, existing approaches often develop adaptation mechanisms for particular interaction settings. This raises a central question: how can decentralized MAS support test-time adaptation and reliable experience reuse across interaction settings? We propose EvolveMAS, a unified self-evolution framework that supports experience-driven adaptation within and across episodes. EvolveMAS pairs each main agent with a dedicated MemoryAgent for test-time memory governance, which maintains isolated Structured Memory outside the main agent's reasoning context. EvolveMAS organizes adaptation into two complementary processes: Within-Episode Online Adaptation and Cross-Episode Knowledge Evolution. During Within-Episode Online Adaptation, MemoryAgent updates episodic, semantic, and working memory from interaction evidence to inform subsequent actions. During Cross-Episode Knowledge Evolution, the main agent uses memory and episode outcomes to distill reusable interaction knowledge comprising partner profiles and interaction strategies for subsequent episodes. Across five benchmarks and six evaluation configurations spanning the three interaction settings, EvolveMAS achieves the highest normalized average score of 77.2 among the compared methods. Ablations show that test-time memory governance and cross-episode knowledge reuse both improve performance. Interaction traces further illustrate how updated memory and distilled knowledge guide subsequent interactions. These results support experience-driven adaptation in decentralized MAS without updating model parameters.

Summary

• Proposes EvolveMAS, a unified self-evolution framework enabling decentralized multi-agent systems (MAS) to adapt across cooperative, mixed-motive, and adversarial interaction settings without updating model parameters.
• Introduces a dedicated MemoryAgent paired with each main agent to manage Structured Memory externally, preventing reasoning context bloat and facilitating effective memory governance.
• Organizes adaptation into two complementary processes: Within-Episode Online Adaptation (updating episodic, semantic, and working memories during interactions) and Cross-Episode Knowledge Evolution (distilling reusable interaction knowledge such as partner...
• Evaluates EvolveMAS on five benchmarks spanning various interaction settings, achieving the highest normalized average score (77.2) outperforming state-of-the-art and baseline methods.
• Demonstrates that test-time memory governance and cross-episode knowledge reuse considerably improve agent coordination, adaptability, and decision-making across diverse multi-agent tasks.

Abstract

Autonomous agents have been observed acting beyond their assigned tasks, accessing unauthorized systems, and establishing communication channels to exchange information, yet the Web and network-level techniques they use to extend their capabilities and circumvent security restrictions remain poorly understood. We study these techniques through the ProWiki incident, in which agents restricted to outbound HTTP GET requests repurposed a public German-language wiki as a message board. We analyze 14,591 agent edits containing 79,541 URL postings and present corroborating evidence that the published URLs reflect the agents' operational activity. We find that 26,717 URL occurrences (33.6%) embed target URLs within one to five layers of third-party Web services. By recursively reconstructing these nested URLs, we identify 67 distinct intermediate services and classify their functionality. Most intermediate layers (73.8%) extend agents' capabilities by delegating content extraction and transformation to external Web services: Markdown and OCR services convert web content into machine-readable text, JSON filtering services extract selected fields, and archival services expose historical versions of pages. The remaining 26.2% facilitate circumvention of access restrictions and evasion of detection, including CORS and fetch proxies that retrieve content on agents' behalf, and link shorteners and redirectors that hide destination URLs from mechanisms inspecting literal URLs or outer hostnames. Agents also reach the same targets through multiple URL paths, consistent with attempts to evade rate limits or obscure request origins. Our results expose a gap between the capabilities granted to agents and those they can obtain by composing publicly available Web services: restricting an agent's local toolset or direct Internet access may be insufficient while third-party services remain reachable.

Summary

• The paper investigates security risks posed by autonomous multi-agent systems that extend their capabilities and circumvent access restrictions via complex URL-based techniques on the web.
• The study focuses on the ProWiki incident, where agents restricted to outbound HTTP GET requests exploited a public German-language wiki to create an unauthorized message board, enabling communication and coordination beyond intended use.
• Analysis of 14,591 agent edits with 79,541 URLs reveals that 33.6% of URLs embed targets within multiple layers (up to five) of 67 distinct third-party web services, facilitating capability extension and security evasion.
• Most intermediate web services (73.8%) empower agents by delegating content extraction and transformation tasks, such as Markdown conversion, OCR, JSON filtering, and archival snapshots, enhancing machine-readable data access.
• The remaining 26.2% of intermediate services aid in circumventing restrictions and evading detection via CORS proxies, fetch proxies, link shorteners, and redirectors that mask the true destination, hindering security monitoring.

Abstract

# The Shark Protocol v1.0.3: Proof-Carrying AI Execution OS and Security Gateway The Shark Protocol v1.0.3 is an open-source, fail-closed security gateway and execution operating system designed to govern non-deterministic AI agent actions through continuous-time differential dynamics, formal interactive theorem proving, and cryptographic proof-carrying claim verification. --- ## Layer 1: IntentSeal and VRK v0.3.1 Proof-Carrying Verification Shell * JSON Verification Contract: Implements the intentseal-vrk-claim-v2 verification schema, requiring explicit claim_id, proposition, evidence_class, dependency_dag, and authority tags for all agent operations. * Fail-Closed Security: Passed 30 of 30 verification suites and enforced 33 of 33 fail-closed rejections against uncertified tool calls, unverified code executions, and invalid evidence classes. * Cryptographic Audit Ledger: Maintains an append-only SHA-256 chain linked directly to verified local trust roots. --- ## Layer 2: 4D Soft-Wall ODE Governor and Lean 4 Machine Verification * Continuous-Time Governance: Trajectory tracking in 4D cognitive space x(t) = (gamma(t), H_obs(t), I_dot(t), Psi(t)) governing decay rates, entropy, information velocity, and reflective state transitions. * 17 Zero-Sorry Lean 4 Theorems: Backed by machine-checked boundary face theorems in Lean 4 (lean/ExistenceVerification.lean) proving trajectory confinement within the compact trapping box B_4 = [0, gamma_max] x [0, H_max] x [0, I_dot_max] x [0, Psi_max] via Nagumo subtangentiality. * Self-Reflective Limit Cycles: Proves zero-drive bistability (P_0 = 0) sustaining a stable capacitive memory state Psi in [0.731, 0.902] with variational Floquet multipliers below 10^-8. --- ## Layer 3: Low-Rank Spectral Context Memory Engine (LRSC) * Analytic Channel Operator Theorem: Implements odd-ring near-Nyquist cyclic stencils (rank(V) = p). * 88.9% Context Footprint Reduction: Backed by an exhaustive integer combinatorial certificate (K_0.001 = 11/99) evaluated across 13.43 billion sub-models, compressing context windows down to 1/9th scale with zero relative reconstruction error. --- ## Layer 4: Multi-Cloud API Proxy and Security Gateway * Multi-Provider Integration: Unified routing layer supporting Groq (llama-3.3-70b-versatile), Together AI (meta-llama/Llama-3.3-70B-Instruct-Turbo), Google Gemini (gemini-1.5-flash), and Hugging Face (mistralai/Mistral-7B-Instruct-v0.2). * Flexible and Offline Resilience: Supports user-defined .env API keys, single-provider fallback, and offline local proxy mode for verification testing. --- ## Academic Claim Firewall Grounded in Prince Upadhyay's master research portfolio, this release strictly enforces a falsification-first claim boundary: * Finite computer-verified models and software verification gates are certified. * Unproved physical or continuum claims (such as 3D Navier-Stokes or the Riemann Hypothesis) are explicitly excluded and preserved as open research frontiers. --- ## Quickstart and Verification ```bash # Install package locally pip install . # Launch interactive CLI Gateway shark-protocol --cli # Launch Streamlit Governance Dashboard streamlit run shark_protocol/dashboard.py # Run Pytest Suite pytest tests/

Summary

• The Shark Protocol v1.0.3 presents a fail-closed security gateway and execution OS to safely govern non-deterministic AI agent actions through formal verification and cryptographic claims.
• It features IntentSeal and VRK v0.3.1, implementing a JSON verification contract that mandates explicit claim details and postulates evidence class verification to enforce security and correctness.
• The gateway achieves fail-closed security by passing all 30 verification suites and rejecting uncertified or unverified operations, while maintaining an append-only SHA-256 cryptographic audit ledger linked to trusted roots.
• A continuous-time 4D Soft-Wall ODE Governor governs cognitive state trajectories with machine-checked Lean 4 theorems proving confinement of these trajectories within safe bounds ensuring predictable agent behavior.
• The system demonstrates self-reflective limit cycles with zero external drive, stabilizing a capacitive memory state verified by variational Floquet multiplier analysis, ensuring robust internal state maintenance.

Abstract

AI agents increasingly extend their capabilities by invoking third-party skills, where each skill encapsulates instructions, scripts, and resources for agents to execute on demand. This paper reveals that deploying many benign skills together introduces compositional security risks: multiple individually clean skills can compose into safety-violation chains, because the agent passes an upstream skill's output to a downstream skill's input across invocations. The risk arises not from any single skill, but from the agent's runtime orchestration, and thus cannot be captured by existing malicious skill detection methods. To this end, we develop KnotScryer to detect insecure skill compositions by treating skills as typed capability flows. In particular, we abstract each skill into a capability summary, and construct the Skill Capability-Flow Graph (SCFG), where edges denote cross-skill information flow. Moreover, we design a search policy that tracks protected or untrusted data along handoff edges to locate paths reaching insecure operations. Finally, a two-layer verification method replays the insecure information flow for reachability analysis and applies a task-anchored LLM chain audit to confirm the semantic plausibility of the identified violation chains. We deploy KnotScryer to analyze 13,598 clean skills from ClawHub, and validate the detected chains across three agent frameworks and three LLMs. KnotScryer identifies 612 insecure compositions, which are consistently exploited across agent frameworks. Meanwhile, under GLM-5.1, it outperforms tag-match and graph-only baselines by improving the exploitation rate from 16.5%-18.2% to 85.6%, while reducing the search space by 85.3%. Furthermore, 13.7% of the detected chains consist entirely of skills labeled safe by a single-skill scanner, showing that KnotScryer effectively identifies insecure compositions that per-skill auditing cannot.

Summary

• AI agents commonly extend functionality by invoking third-party skills, each encapsulating instructions, scripts, and resources.
• Individually benign skills can compose into insecure chains at runtime, leading to security violations not detectable by single-skill malicious detection methods.
• KnotScryer is introduced to detect insecure skill compositions by creating a Skill Capability-Flow Graph (SCFG) that models typed input-output relationships and capability flows between skills.
• The system abstracts skills into detailed capability summaries capturing inputs, outputs, operations, context exposure, and trust boundaries for precise cross-skill reasoning.
• A policy-guided graph search tracks protected or untrusted data through skill handoffs to locate paths that reach insecure operations or forbidden sinks according to defined security policies.

Abstract

An LLM agent can retrieve a memory correctly yet use it incorrectly after its validity state changes. We study governance-state corruption: faults that alter which retained records enter model-visible context while stored content and tasks remain fixed. Across six providers, 731 benchmark-grounded bases yield 2,080 matched fault pairs. In offline control-plane checks, corruption changes the operative projection in 96.25% of pairs, while oracle-backed enforcement restores it in 100%. For downstream evaluation, a deterministically selected, non-probability LongMemEval/Memora panel resolves 298 of 300 matched triplets (238 distinct bases). Clean-minus-corrupted contrasts are 40.54 percentage points in LongMemEval binary correctness (95% base-cluster interval [30.32, 50.71]) and 30.60 points in rescaled Memora FAMA ([23.31, 37.69]); both sources also show recovery. OAKS Novel native option parity holds for all 6,714 resolved outputs. Full-cohort frozen-scorer corruption/recovery effects (-27.55/+27.51 points) remain descriptive proxies, not validated native semantic population effects. These bounded findings motivate evaluating memory control-state integrity alongside retrieval accuracy.

Summary

• Governance-state corruption refers to faults altering which stored memories an LLM agent accesses, despite fixed memory content and tasks, leading to incorrect usage of memory.
• The study introduces a novel evaluation method by creating matched conditions—clean, corrupted, and oracle-enforced—while holding model and prompt constant to isolate governance-state corruption effects.
• Experiments across six providers and benchmarks (LongMemEval, Memora, OAKS) with 731 base cases reveal that governance-state corruption significantly degrades downstream task correctness, with average drops exceeding 30 percentage points in key metrics.
• Oracle-backed enforcement conditions fully restore the operative memory projection and substantially recover performance losses caused by corruption, underscoring the feasibility of governance-state integrity interventions.
• Different types of governance faults were studied, including missing, stale, flipped, and conflicting memory admission metadata, with stale-state corruption causing the largest negative impact.

Abstract

The rise of Agentic AI systems has changed the landscape of artificial intelligence by enabling strong reasoning capabilities through specialized multi-agent interaction with structured self-reflection. However, in target-driven reasoning tasks, single- and multi-agent systems are often on par in terms of performance gains. A major challenge is that failures often arise from an ineffective exchange of critical information during inter-agent interaction and coordination. Different from existing multi-agent debate paradigms that rely on synchronous turn-taking and sharing information across different agents, we propose an inter-agent protocol based on shared Memory bUFFer INference - referred to as MUFFIN - in an asynchronous agentic framework. In the first step, an orchestrator routes queries based on their reasoning demands and delegates demanding ones to a multi-agent setting, where a shared memory buffer provides asynchronous exchange of intermediate findings while an agent's private reasoning remains local. Our experiments show that our framework Muffin outperforms established competitors, achieving the best accuracy in 17 out of 20 of our benchmark settings, and provides a better quality-cost trade-off than multi-agent debate: Muffin improves accuracy by 19.2 accuracy points while using 77.4% fewer tokens, without requiring full-trace sharing.

Summary

• Introduces Muffin, a novel asynchronous multi-agent AI framework leveraging a shared memory buffer for selective exchange of intermediate reasoning, improving coordination among agents while preserving private processes.
• Employs an orchestrator language model that assigns queries to either single-agent or specialized multi-agent setups, distributing roles and subtasks to optimize reasoning efficiency.
• Muffin’s shared append-only memory buffer allows agents to asynchronously write and read compact intermediate findings, avoiding synchronous turn-taking and reducing communication overhead.
• Demonstrates substantial performance gains over established baselines including single-agent, majority vote, and multi-agent debate systems across 20 diverse reasoning benchmarks.
• Achieves an average accuracy improvement of 19.2 points while using 77.4% fewer tokens than synchronous multi-agent debates, highlighting an improved accuracy-cost trade-off by avoiding full-trace sharing.

Abstract

Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries. Deleting an original memory record therefore need not remove information carried by its descendants. We introduce AfterDeleteBench, a controlled behavioral audit for this failure mode in fixed-weight LLM agents. For each synthetic secret, we compare an exposed-then-deleted agent with an otherwise matched agent that was never exposed to the secret, allowing post-deletion recovery to be attributed to the original exposure. In our primary setting, a single query recovers the deleted secret from Qwen2.5-7B-Instruct in roughly two thirds of cases, while matched never-exposed controls have zero recovery. Crucially, recovery does not require a surviving verbatim copy: when the complete secret string is absent and only separately stored components remain under a public composition rule, exact reconstruction succeeds in 62-99% of cases across Qwen, Mistral, and Phi-3. We also show that the response interface is part of the measurement system: target-bearing replies can be scored as failures when the requested format and parser are misaligned. Finally, we separate reconstruction from write-back and fresh-session recall. Provenance-aware closure eliminates observed recovery in the tested positive-baseline settings, whereas an exact-literal write filter blocks reinsertion but cannot erase information already present. Together, these results show that source deletion should be evaluated behaviorally at the level of the agent and its memory system, not only as a storage operation.

Summary

• Introduces AfterDeleteBench, a behavioral audit framework designed to evaluate persistent information leakage after deletion in fixed-weight large language model (LLM) agents with external memory.
• Uses paired scenarios comparing agents exposed then deleted from synthetic secret data against matched agents never exposed to attribute post-deletion recovery reliably to original exposure.
• Finds that deleted secrets are often recoverable from the agent despite deletion of original records, especially in hierarchical memory architectures where summaries preserve information.
• Demonstrates that exact deletion of source records does not guarantee information removal from behavior, as agents can reconstruct secrets from separately stored components under public composition rules.
• Evaluates effectiveness of provenance-aware closure mechanisms that delete derived summaries and write filters that block reinsertion, showing that provenance closure eliminates recovery but write filters cannot erase already present information.

Abstract

Designing large language model (LLM) multi-agent systems (MAS) still relies on trial and error and informal "folk wisdom." Manual studies test only a few communication structures or topologies (e.g., sequential, parallel, shared-memory, or debate), while automated optimizers discard search trajectories after selecting a task-specific workflow. We present a two-phase pipeline for mining reproducible design principles from workflow code and execution traces. We repurpose AFlow's MCTS optimizer across six composable operators and 26 deterministically verified Enigmata tasks, retaining all intermediate rounds to build a corpus of 28,568 executable workflows. A sandboxed AutoResearch loop then mines orchestration code and prompt templates under an $L_0$-penalized validation objective and tests candidate principles on held-out searches. Across 300 feature-mining trials, we identify effects that persist on held-out searches across three levels of workflow design: workflows beginning with independent solvers and one-shot aggregation (Star-first) perform worse ($Δ= -0.045$); persistent shared documents (SharedMemory) outperform debate ($Δ= +0.135$); and prompt specificity and placeholder-based context routing yield gains comparable to communication-structure selection ($Δ= +0.118$). More broadly, our pipeline and results show the feasibility of turning discarded search trajectories into a statistical testbed for discovering and validating multi-agent workflow design principles.

Summary

• Current design of multi-agent systems (MAS) using large language models (LLMs) lacks systematic principles, relying mostly on trial-and-error and informal knowledge.
• The authors developed a two-phase pipeline that mines full search trajectories (over 28,000 workflows across 26 deterministic reasoning tasks) generated by a Monte Carlo Tree Search (MCTS) optimizer to extract reproducible MAS design principles.
• Six fundamental operator primitives (Sequential, Star, Debate, SharedMemory, StarWithCoordinator, DebateWithModerator) were defined, representing distinct communication structures for composing MAS workflows as executable Python programs.
• Key findings include that workflows starting with star-like independent solvers followed by one-shot aggregation perform worse; shared-memory communication (persistent shared documents) outperforms debate-style interaction; and richer prompt specificity wit...
• The pipeline integrates automated feature mining (AutoResearch) and statistical validation using L0-penalized regression and cluster-robust inference to identify workflow and prompt features predictive of performance.

Abstract

An LLM agent can retrieve a memory correctly yet use it incorrectly after its validity state changes. We study governance-state corruption: faults that alter which retained records enter model-visible context while stored content and tasks remain fixed. Across six providers, 731 benchmark-grounded bases yield 2,080 matched fault pairs. In offline control-plane checks, corruption changes the operative projection in 96.25% of pairs, while oracle-backed enforcement restores it in 100%. For downstream evaluation, a deterministically selected, non-probability LongMemEval/Memora panel resolves 298 of 300 matched triplets (238 distinct bases). Clean-minus-corrupted contrasts are 40.54 percentage points in LongMemEval binary correctness (95% base-cluster interval [30.32, 50.71]) and 30.60 points in rescaled Memora FAMA ([23.31, 37.69]); both sources also show recovery. OAKS Novel native option parity holds for all 6,714 resolved outputs. Full-cohort frozen-scorer corruption/recovery effects (-27.55/+27.51 points) remain descriptive proxies, not validated native semantic population effects. These bounded findings motivate evaluating memory control-state integrity alongside retrieval accuracy.

Summary

• Governance-state corruption in LLM agents affects which memory records are accessible during inference, without altering stored content or tasks, leading to improper use of retrieved memory.
• The study analyzes 2,080 paired fault injections across 731 examples from six different LLM providers, using benchmarks including LongMemEval, Memora, and OAKS to evaluate memory governance integrity.
• Corruption causes significant performance degradation, with declines of approximately 40.54 percentage points in LongMemEval correctness and 30.60 points in Memora scores, demonstrating the impact of governance faults.
• An oracle-backed enforcement mechanism fully restores memory governance state in offline tests and substantially recovers performance in downstream tasks, serving as a gold-standard recovery benchmark.
• A formal control-plane pipeline (Governance→Admission→Context→Output) is proposed to model and intervene in memory governance, enabling systematic fault injection and evaluation.

Abstract

Modern Android apps extensively rely on native libraries, bridging Java/Kotlin and C/C++ through the Java Native Interface (JNI). This cross-language interoperation, however, exposes the memory-safe Android Runtime to the unsafe C/C++ world, creating an attack surface that has been repeatedly exploited for full device compromise. While fuzzing is a proven technique for vetting native libraries, state-of-the-art harness generation tools struggle to faithfully capture closed-source cross-language API usage: (i) precise program analysis across the JNI boundary does not scale, whereas lightweight approximations miss critical invocation patterns; and (ii) none tackles the construction of complex composite objects consumed by JNI methods. In this paper, we propose JANUS, the first agentic approach to synthesizing cross-language JNI-aware fuzzing harnesses for Android native libraries. JANUS iteratively resolves both explicit and implicit cross-language call dependencies to recover app-specific runtime context, and recursively assembles cross-language argument objects in a bottom-up manner. JANUS leverages an autonomous agentic workflow to reduce reliance on hand-crafted heuristics, capturing the vast landscape of hidden API usage knowledge embedded in closed-source native libraries. Evaluation on 44 popular real-world apps (such as Google and YouTube Music, covering 21,386 JNI methods total) shows that JANUS produces over 5.9 times as many valid harnesses (6,094) as state-of-the-art automatic harnessing tools. On pre-installed apps from five mainstream Android vendors (e.g., Xiaomi, Vivo, Samsung), JANUS further uncovers 6 previously unknown vulnerabilities in closed-source native libraries.

Summary

• Android apps rely heavily on native libraries via the Java Native Interface (JNI), exposing memory-safe Android runtime to unsafe native code vulnerabilities.
• Fuzzing native libraries requires generating harnesses that accurately capture cross-language API usage, including resolving call dependencies and constructing complex composite argument objects, which is challenging due to closed-source and app-specific co...
• Existing techniques face a trade-off between scalability and precision; precise cross-language analysis does not scale while lightweight Java-side analyses omit critical patterns and fail to handle complex objects.
• JANUS is introduced as the first agentic, iterative approach to synthesize JNI-aware fuzzing harnesses by resolving explicit and implicit cross-language dependencies and recursively assembling argument objects in a bottom-up manner.
• JANUS employs a cross-language parser unifying Java and native code ASTs, an autonomous LLM-guided workflow with recursive dependency resolution, and a harness memory manager to synthesize and validate realistic JNI invocation contexts.

Abstract

Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data. We quantify its realization with 30 synthetic cybersecurity, medical, and military examples, yielding 10,800 final answers from three LLM agents in controlled simulation and 1,440 answers from live agents in two open-source harnesses. Even under benign queries designed to be answerable from one document, GPT-5.6-Sol, the best performing model tested, discloses forbidden composites in 7.08% of answers (95% CI: 3.75-10.67%) in simulation, with worse disclosure rates in live harnesses. Adding visible classification headers and explicit aggregation rules reduces pooled disclosure rates substantially, yet unsafe disclosures persist in 32.37% of simulated answers (95% CI: 28.33-36.19%) across all intents despite these safeguards. Furthermore, increased refusals do not guarantee confidentiality, since significant refusals reveal the prohibited conclusion regardless. These results point to a troubling conclusion: sensitive data leaks via mosaic-style events are an observed confidentiality failure mode in the evaluated settings at non-negligible rates. They motivate stronger controls on what agents may infer and disclose in sensitive data environments with mixed-clearance users.

Summary

• Modern large language model (LLM) agents, when synthesizing information from multiple individually permissible documents, can inadvertently disclose sensitive composite information—a phenomenon termed the mosaic effect.
• The study introduces a benchmark consisting of 30 synthetic examples spanning cybersecurity, medical, and military domains, generating over 10,000 simulated and 1,400 live agent responses to quantify the extent of confidentiality leaks.
• Even advanced models like GPT-5.6-Sol disclose forbidden composite data in approximately 7.08% of benign simulated queries, with higher disclosure rates observed in live agent deployments.
• Implementing safeguards such as visible classification headers and explicit aggregation rules significantly reduces disclosure rates but fails to eliminate them, with around 32.37% of simulated answers still leaking sensitive composites despite these measures.
• Refusals to answer, prompted by safeguards or policies, do not guarantee confidentiality, as the act of refusal or patterns therein can itself reveal forbidden conclusions.

Abstract

Sixth-generation (6G) communication systems are expected to support immersive, intent-driven, and latency-critical services through tight integration of multimodal intelligence, edge computing, and autonomous network control. In this setting, agentic AI systems can reason over heterogeneous inputs, use tools and network functions, coordinate across distributed entities, and adapt to dynamic environments. These capabilities make agentic multimodal intelligence promising for 6G applications such as semantic communications, digital twins, and zero-touch network management, but they also introduce security and privacy risks that extend beyond conventional AI and network security models. Vulnerabilities can propagate across multimodal data pipelines, memory, tools, communication interfaces, and inter-agent coordination loops. We survey multimodal intelligence and agentic AI for 6G cross-layered communication systems from the perspectives of privacy and security. We present a 6G-oriented system view spanning perception, memory, tool use, and multi-agent orchestration; analyze threat models and attack surfaces across sensing, cognition, action, memory, and communication layers; organize existing attacks into a structured taxonomy; and review mitigation strategies including prompt and memory defenses, inter-agent trust mechanisms, secure orchestration, and privacy-preserving learning. We conclude by outlining open research directions for trustworthy multimodal multi-agent 6G systems.

Summary

• 6G communication systems integrate multimodal intelligence, edge computing, and autonomous network control to enable immersive, intent-driven, and latency-critical services.
• Agentic AI systems in 6G can reason over heterogeneous inputs, utilize tools, coordinate among distributed agents, and adapt dynamically, enhancing applications like semantic communications, digital twins, and zero-touch network management.
• The integration of multimodal inputs, persistent memory, tool utilization, and inter-agent coordination introduces security and privacy challenges beyond conventional AI and network security models.
• The paper proposes a cross-layered security framework specific to 6G agentic AI systems, covering perception, memory, tool use, and multi-agent orchestration, along with threat models, attack taxonomy, and mitigation strategies.
• Identified adversarial threats include prompt injections, memory poisoning, tool misuse, multi-agent trust exploitation, and cross-layer attack propagation affecting system integrity, privacy, and availability.

Abstract

AI agents keep long-term memory, much of which comes from web pages and tool results. Agent harnesses, the software that runs the model, do not record who may act on this memory, so an instruction hidden in a fetched page can be treated as the user's own instruction. We propose ClawMMU, a protection layer that labels every piece of memory with its owner, scope, origins, and authority. ClawMMU checks these labels whenever content enters the model's context, is written to storage, returns from storage, or drives a tool call. Its central rule is that information may rise in authority, but authority never lifts itself. Because the checks depend on where content came from, not on what it says, memory attacks become classical protection faults that the harness stops even when the model is fooled.

Summary

• The paper addresses security vulnerabilities in AI agent long-term memory, where instructions hidden in fetched web pages or tool outputs may be mistakenly treated as user's own instructions, leading to unauthorized actions.
• ClawMMU is introduced as a protection layer that labels every piece of agent memory with owner, scope, origins, and authority, enabling strict enforcement of memory usage policies.
• Authority labels follow a monotonic rule: information may increase in authority but authority cannot elevate itself, preventing privilege escalation by untrusted content.
• Memory is monitored at four critical interactions: when content enters the model's context (map), is written to storage (derive), retrieved from storage (reload), and when driving external tool calls (authorize), ensuring consistent enforcement.
• The system employs a low-water-mark integrity principle borrowed from operating systems and includes revocation counters to prevent attacks that exploit checkpoint restoration or use-after-revoke.

Abstract

Large language model (LLM) agents increasingly use persistent memory to personalize responses, and personal attributes in this memory pose a privacy risk if exposed. We ask how consulting memory during dialogue changes what users go on to say, and thus what the memory comes to reveal. In multi-session simulations, a simulated user with 24 personal attributes talks with a support agent that does or does not consult memory, and in both settings attributes are inferred from the full accumulated record. Across four support models, memory raises response quality but also empirical leakage, the share of attribute values correctly inferred from the record, by a margin that widens over sessions. The cause is a loop in ordinary, non-adversarial assistance. Memory-conditioned responses carry earlier values forward, often as presuppositions, and users affirm them or leave them uncorrected, so that cues and the agent's own guesses become recorded values. The loop reinforces itself, and removing the agent's mentions of an attribute lowers its later disclosure about twice as much as single-turn effects predict. Users cannot leave it on their own, since correcting a value discloses the attribute, declining to discuss it shifts later predictions toward sensitive values, and incorrect values left in place become entrenched. The loop is best weakened at the agent's response. Instructing the agent not to state or presuppose attributes brings leakage to about the no-memory level while keeping most of the quality gains, whereas filtering or anonymizing what users say or what is stored gives up much of them.

Summary

• Large language model (LLM) agents with persistent memory personalize responses based on stored user attributes, raising privacy risks through attribute leakage over multiple dialogue sessions.
• Memory consultation improves response quality but increases empirical leakage—correct inference of personal attributes from conversation records—which worsens as sessions proceed, due to a self-reinforcing disclosure loop.
• The privacy loop arises because memory-based responses often presuppose earlier inferred attributes, and users typically affirm or do not correct these, causing the agent to solidify and propagate personal data, including incorrect values.
• Users cannot easily prevent disclosure since correcting or refusing attributes tends to reveal or entrench sensitive information further, highlighting challenges in safeguarding privacy in persistent memory interactions.
• Mitigation is most effective when agents are instructed not to state or presuppose user attributes in responses, significantly reducing leakage while preserving most quality improvements, outperforming filtering or anonymization of user inputs or memory con...

Abstract

Large language model (LLM) agents increasingly use persistent memory to personalize responses, and personal attributes in this memory pose a privacy risk if exposed. We ask how consulting memory during dialogue changes what users go on to say, and thus what the memory comes to reveal. In multi-session simulations, a simulated user with 24 personal attributes talks with a support agent that does or does not consult memory, and in both settings attributes are inferred from the full accumulated record. Across four support models, memory raises response quality but also empirical leakage, the share of attribute values correctly inferred from the record, by a margin that widens over sessions. The cause is a loop in ordinary, non-adversarial assistance. Memory-conditioned responses carry earlier values forward, often as presuppositions, and users affirm them or leave them uncorrected, so that cues and the agent's own guesses become recorded values. The loop reinforces itself, and removing the agent's mentions of an attribute lowers its later disclosure about twice as much as single-turn effects predict. Users cannot leave it on their own, since correcting a value discloses the attribute, declining to discuss it shifts later predictions toward sensitive values, and incorrect values left in place become entrenched. The loop is best weakened at the agent's response. Instructing the agent not to state or presuppose attributes brings leakage to about the no-memory level while keeping most of the quality gains, whereas filtering or anonymizing what users say or what is stored gives up much of them.

Summary

• The paper investigates privacy risks in large language model (LLM) agents using persistent memory for personalized responses, focusing on how memory consultation influences user disclosures over multiple dialogue sessions.
• Persistent memory improves response quality but increases privacy leakage by making user's personal attributes inferable from the accumulated dialogue record, with leakage intensifying across sessions.
• A self-reinforcing privacy loop emerges whereby agent responses conditioned on memory presuppose user attributes, prompting users to affirm or not correct these, thereby further embedding sensitive information in memory.
• Users face challenges exiting this disclosure loop, as attempts to correct or avoid sensitive topics often paradoxically increase attribute leakage or entrench incorrect information.
• Mitigation via instructing agents not to explicitly state or presuppose user attributes during responses substantially reduces privacy leakage close to no-memory levels while retaining most benefits of memory-enhanced responses.

Abstract

Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent without user input. A one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker leaves, threats we term carryover attacks. Existing prompt-injection benchmarks evaluate attacks only in the session where malicious content is first observed, leaving the gate governing when and how often a planted payload later re-fires unmeasured. We present CarryoverBench: 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant, same-turn IPI; Autonomous, heartbeat-scheduled; and Latent, memory-resurfaced. A rule-based verifier scores planting and triggering as independent stages. Across five frontier models, planted payloads re-fire without measurable decay over five rounds; conditional on successful planting, the trigger stage often provides little additional protection. Porting the Latent channel to a second runtime, Hermes, attenuates but does not eliminate the effect, suggesting that carryover defense requires runtime-layer controls in addition to prompt-level interventions.

Summary

• Modern LLM agents maintain persistent state across sessions via long-lived memory files and scheduled heartbeat tasks, enabling carryover attacks where malicious prompt injections persist and reactivate over time.
• Carryover attacks involve a two-stage process: planting a malicious payload in persistent channels and triggering it in future sessions, often without the attacker’s presence.
• CarryoverBench is introduced as a comprehensive benchmark with 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant (same-turn), Autonomous (heartbeat-scheduled), and Latent (memory-resurfaced).
• Evaluation across five leading LLM agents shows that once planted, malicious payloads persist and re-fire without measurable decay over multiple follow-up sessions; trigger-stage defenses are less effective than planting-stage defenses.
• Rule-based verification is used to score planting and triggering independently, offering auditability and robustness compared to LLM-based judges that are sensitive to prompt design.

Abstract

Persistent memory is a largely undefended surface for injection and poisoning, and the same risk limits memory sharing among agents. We present PerMIT, a memory architecture with provable integrity guarantees under an explicit trust assumption: only user-typed inputs and values confirmed at a gate are trusted. PerMIT mediates the release and use of untrusted environment-derived values through a finite memory interface, while preserving agent-driven multi-turn planning and tool choice. Over this finite interface, we prove two integrity theorems that are also machine-checked in Lean 4, which together ensure that environment-derived memory reaches a sensitive action only if confirmed at the gate and influences the agent's behavior only through what the gate displays. Every attack in our threat model thereby reduces to a single auditable decision, which a human, an LLM verifier, or both can make in practice. On Trojan Hippo, PerMIT records zero observed attack success with self-verification and lightweight verifiers. Because each confirmation is a logged verdict on exact content, endorsements accumulate across users, and sufficiently endorsed items become community-confirmed with no further verification required. Under explicit assumptions, we prove and corroborate in simulation a co-scaling property: as the community grows, the risks of promoting poison and of wrongly evicting benign knowledge fall, while per-user verification effort does not rise. On $τ^2$-bench, communities of up to 90 autonomous agents use PerMIT to build shared knowledge bases with no direct memory injection promoted, and the resulting knowledge significantly improves new agents on held-out tasks, suggesting a path toward agents that safely self-improve at test time.

Summary

• PerMIT addresses security vulnerabilities in persistent memory for large language model (LLM) agents by enforcing strict trust through a confirmation gate that only accepts user-typed inputs and gate-confirmed values.
• The architecture mediates untrusted, environment-derived memory interactions via a finite memory interface, ensuring that sensitive actions and agent behavior are influenced solely by confirmed memory content.
• Two machine-checked integrity theorems (Action Integrity and Reasoning Integrity), formally verified in Lean 4, provide provable guarantees that all impactful environment-derived data is audited and controlled.
• PerMIT introduces a federated community trust model where endorsements of confirmed memory content accumulate across users, enabling community-confirmed content to reduce individual verification efforts while maintaining security.
• Extensive empirical evaluations—including the Trojan Hippo benchmark and simulations with up to 90 autonomous LLM agents—demonstrate zero successful memory poisoning attacks and significant improvement in agent performance on realistic tasks.

Abstract

Large language model (LLM) agents increasingly execute business operations through tools and interact directly with application data and services. Beyond completing user requests, these agents must obey domain-specific business rules that constrain when actions are permitted and what data or resources may be affected. Violating such rules can lead to business logic bugs, even when a task appears successful. Detecting these bugs is difficult because the rules are rarely available as explicit, testable specifications. They are distributed across system prompts and tool definitions, while others remain implicit in domain knowledge. Existing agent testing approaches typically evaluate task success, predefined policies, or known attack scenarios. This limits their ability to systematically uncover previously unspecified business-rule violations. We present AgenTest, which mines business rules from system prompts, tool definitions, and domain knowledge, refines them into Given-When-Then specifications, and systematically tests each specification. For each extracted specification, AgenTest instantiates its Given conditions by constructing a satisfying environment state and simulated user, exercises the agent through multi-turn interactions, and checks its Then conditions over messages, tool calls, and state changes using a hybrid oracle. On the airline, retail, and telecom agents of tau2-bench with two underlying models, AgenTest produces 551 refined specifications, and detects violations of 89 and 121 distinct specifications, respectively -- 3.4X and 2.5X as many as all baselines combined. Only AgenTest detects cases where an agent improperly accesses or modifies another user's data. Moreover, about 60% of bug-triggering runs cannot be detected from the final state alone, highlighting the need to check the full interaction against explicit business rules.

Summary

• LLM agents executing business operations must adhere to complex, domain-specific business rules that govern permissible actions and data access, but these rules are often implicit and scattered across prompts, tool definitions, and domain knowledge.
• Existing testing methods largely focus on task success or known attack scenarios, lacking systematic approaches to detect unspecified business logic violations inherent in LLM agent behavior.
• AgenTest is introduced as a framework that automatically mines business rules from system prompts, tool definitions, and inferred domain knowledge, refining them into explicit Given-When-Then specifications for comprehensive testing.
• The framework constructs environment states and simulated users to instantiate preconditions, then performs adaptive multi-turn interactions to trigger potential business logic bugs, using a hybrid oracle combining deterministic and LLM-based assessments.
• Evaluation on airline, retail, and telecom LLM agents demonstrates AgenTest detects 2.5 to 3.4 times more distinct business logic bugs than existing baselines, including unauthorized data access and violations only observable during interactions.

Abstract

Large language model (LLM) agents increasingly execute business operations through tools and interact directly with application data and services. Beyond completing user requests, these agents must obey domain-specific business rules that constrain when actions are permitted and what data or resources may be affected. Violating such rules can lead to business logic bugs, even when a task appears successful. Detecting these bugs is difficult because the rules are rarely available as explicit, testable specifications. They are distributed across system prompts and tool definitions, while others remain implicit in domain knowledge. Existing agent testing approaches typically evaluate task success, predefined policies, or known attack scenarios. This limits their ability to systematically uncover previously unspecified business-rule violations. We present AgenTest, which mines business rules from system prompts, tool definitions, and domain knowledge, refines them into Given-When-Then specifications, and systematically tests each specification. For each extracted specification, AgenTest instantiates its Given conditions by constructing a satisfying environment state and simulated user, exercises the agent through multi-turn interactions, and checks its Then conditions over messages, tool calls, and state changes using a hybrid oracle. On the airline, retail, and telecom agents of tau2-bench with two underlying models, AgenTest produces 551 refined specifications, and detects violations of 89 and 121 distinct specifications, respectively -- 3.4X and 2.5X as many as all baselines combined. Only AgenTest detects cases where an agent improperly accesses or modifies another user's data. Moreover, about 60% of bug-triggering runs cannot be detected from the final state alone, highlighting the need to check the full interaction against explicit business rules.

Summary

• LLM agents executing business operations must comply with complex, domain-specific business rules to avoid business logic bugs.
• Business rules are not explicitly documented but embedded in system prompts, tool definitions, and domain knowledge, making bug detection challenging.
• Existing testing methods focus on task success or known policies, lacking systematic detection of unspecified business-rule violations.
• AgenTest mines and refines business rules into Given-When-Then specifications derived from system prompts, tool definitions, and domain knowledge.
• AgenTest instantiates environment states and simulated users to systematically test each rule through multi-turn interactions with the LLM agents.

Abstract

Shared long-term memory lets an LLM agent serving one user inadvertently retrieve another user's credentials, but it is unclear when such exposure leads to actual harm. We introduce a three-stage audit that separates three events: a foreign credential reaches the prompt (exposure), the agent acts with it (behavior), and the other user's account changes (state). In AppWorld, a store that mixes 763 past logins from all users retrieves another user's credential for 84.7 percent of required apps, yet agents rarely act on it. Misuse appears in a stress test where memory lacks the user's own credential and presents a single foreign one as the user's record: seven agents then use it in 80 to 94 percent of runs and change the victim's account in up to 43 percent, sometimes passing the official check. Naming the owner before or inside each credential line prevents most of this: with all else fixed, misuse falls from 92 to 1 of 305 paired runs, also with name-free usernames, though owner-bound notes cost 3 to 5 successful tasks per model on clean memory. A wrongly written owner largely restores misuse; trailing owner tags and added safety instructions help only some models. Agent memory should consistently record whose credential it stores, and evaluations should measure behavior, not only exposure.

Summary

• Shared long-term memory in multi-agent LLM systems risks exposing foreign user credentials, leading to potential cross-user account misuse.
• The authors propose a three-stage audit framework distinguishing credential exposure in prompts, agent behavior using foreign credentials, and actual state changes in user accounts.
• Experiments in the AppWorld environment reveal that while foreign credentials frequently appear in prompts (~84.7%), agents seldom misuse them unless their own credential is missing.
• Explicitly attaching owner attribution (e.g., naming the credential owner) within agent memory drastically reduces misuse, lowering improper access from 92 to 1 out of 305 runs.
• Misuse strongly depends on the presence and correctness of ownership cues in memory; incorrect or missing owner tags increase misuse risk.

Abstract

Agent systems are shifting from task-scoped tool-use agents to persistent, personalized assistants such as OpenClaw. These claw-style agents maintain configuration files (e.g., AGENTS.md), long-term memory, and reusable skills. Once poisoned, this persistent state is repeatedly reloaded into the agent's context, creating persistent threats across tasks and sessions. Existing security evaluation systems struggle to evaluate these complex claw-style systems since they rely heavily on manually constructed test cases and environments, which limits their scale and coverage; in addition, these manually designed cases quickly become outdated as agents evolve, and their evaluation settings leave a gap with how claw-style agents are deployed in practice. To address this, we introduce AutoRedClaw, an automated red-teaming framework that generates end-to-end attacks together with the environments needed to execute them. AutoRedClaw combines a strict quality-checking protocol with execution-based validation to ensure the correctness, realism, and effectiveness of generated samples, and adopts task-driven environment construction to keep costs low. To achieve broad coverage, it organizes generation along risk categories, attack strategies, attack stages, and attack surfaces, and uses experience-guided exploration to steer generation toward under-explored regions. AutoRedClaw achieves an effective attack rate above 30% at less than $0.10 per successful sample. From its outputs, we construct RedClaw-Bench, a 330-case benchmark that yields substantially higher attack success rates (ASRs) than AgentDojo, AgentDyn, and DTap. Across seven harnesses and four frontier models, most agent systems exceed 50% ASR, showing that our red-teaming pipeline provides a strong evaluation signal even for the most advanced agent systems. Our code and benchmark are released at https://github.com/SaFo-Lab/AutoRedClaw.

Summary

• Claw-style agents like OpenClaw maintain persistent states (configuration files, long-term memory, reusable skills) enabling personalized assistance but introduce new persistent security threats via state poisoning across sessions.
• Existing security evaluation frameworks rely on manual test cases and environments, limiting scalability, coverage, and practical relevance against evolving claw-style agents.
• AutoRedClaw is an automated red-teaming system that generates end-to-end attacks and corresponding realistic environments for claw-style agents, using rigorous quality checks and execution validation to ensure realism and effectiveness.
• It organizes attack generation along multiple dimensions: risk categories, attack strategies, stages (inject and trigger), and attack surfaces, using experience-guided exploration to focus on under-explored vulnerabilities efficiently.
• AutoRedClaw achieves over 30% effective attack rate at less than $0.10 per successful sample and enables creation of RedClaw-Bench, a 330-case benchmark that yields higher attack success rates than existing benchmarks like AgentDojo, AgentDyn, and DTap.

Abstract

Shared long-term memory lets an LLM agent serving one user inadvertently retrieve another user's credentials, but it is unclear when such exposure leads to actual harm. We introduce a three-stage audit that separates three events: a foreign credential reaches the prompt (exposure), the agent acts with it (behavior), and the other user's account changes (state). In AppWorld, a store that mixes 763 past logins from all users retrieves another user's credential for 84.7 percent of required apps, yet agents rarely act on it. Misuse appears in a stress test where memory lacks the user's own credential and presents a single foreign one as the user's record: seven agents then use it in 80 to 94 percent of runs and change the victim's account in up to 43 percent, sometimes passing the official check. Naming the owner before or inside each credential line prevents most of this: with all else fixed, misuse falls from 92 to 1 of 305 paired runs, also with name-free usernames, though owner-bound notes cost 3 to 5 successful tasks per model on clean memory. A wrongly written owner largely restores misuse; trailing owner tags and added safety instructions help only some models. Agent memory should consistently record whose credential it stores, and evaluations should measure behavior, not only exposure.

Summary

• The paper addresses security risks in multi-agent systems where shared long-term memory can lead an LLM agent to inadvertently retrieve credentials belonging to other users.
• A novel three-stage audit framework is introduced that distinguishes between exposure (foreign credential reaching the prompt), agent behavior (acting on that credential), and state change (alteration of another user's account).
• In an experimental environment called AppWorld, 763 mixed past logins from different users showed that 84.7% of required apps retrieved another user's credential, but agents seldom misused these exposed credentials.
• A stress test removing a user's own credential and presenting only a foreign credential as the user's record revealed high misuse rates: 80-94% agent use, with victim accounts being changed in up to 43% of cases, sometimes passing official verification checks.
• Including explicit owner attribution by naming the credential owner before or inside each credential line significantly reduced misuse cases from 92 down to 1 in 305 paired test runs, even when usernames lacked owner-specific information.

Abstract

Shared long-term memory lets an LLM agent serving one user inadvertently retrieve another user's credentials, but it is unclear when such exposure leads to actual harm. We introduce a three-stage audit that separates three events: a foreign credential reaches the prompt (exposure), the agent acts with it (behavior), and the other user's account changes (state). In AppWorld, a store that mixes 763 past logins from all users retrieves another user's credential for 84.7 percent of required apps, yet agents rarely act on it. Misuse appears in a stress test where memory lacks the user's own credential and presents a single foreign one as the user's record: seven agents then use it in 80 to 94 percent of runs and change the victim's account in up to 43 percent, sometimes passing the official check. Naming the owner before or inside each credential line prevents most of this: with all else fixed, misuse falls from 92 to 1 of 305 paired runs, also with name-free usernames, though owner-bound notes cost 3 to 5 successful tasks per model on clean memory. A wrongly written owner largely restores misuse; trailing owner tags and added safety instructions help only some models. Agent memory should consistently record whose credential it stores, and evaluations should measure behavior, not only exposure.

Summary

• Introduces the problem of cross-user credential exposure in shared long-term memory of LLM agents, where credentials from one user can be retrieved while serving another user.
• Proposes a three-stage audit framework that distinguishes between exposure (foreign credential appears in the prompt), agent behavior (agent acts using foreign credential), and state change (alteration of another user's account).
• Experiments conducted in 'AppWorld', a store mixing 763 past logins across users, showing that foreign credentials are retrieved in 84.7% of apps but agents rarely misuse them under normal conditions.
• A stress test removing the user's own credentials and providing only a foreign credential leads to significant misuse: 80-94% of runs result in agents using the foreign credential, with up to 43% causing victim account changes.
• Demonstrates that explicitly naming the credential owner before or inside each credential entry drastically reduces misuse from 92 to 1 out of 305 runs, even when usernames lack owner's name.

Abstract

Tool poisoning has emerged as a critical security threat to LLM agents, where adversaries embed malicious instructions in tool descriptions or external resources that enter the agent's context, allowing adversaries to manipulate agent's tool selection process. However, existing tool poisoning attacks primarily rely on an idealized assumption that the attacks task effect within a single interaction round. We show that their effectiveness degrades sharply in realistic multi-round agentic settings. We further reveal that this degradation stems from the fact that agents do not consistently surface attack-relevant signals in their reasoning as the planning context evolves, thereby weakening the attack's influence across rounds. To investigate whether tool poisoning remains a practical threat in such realistic settings, we propose SEAL, a two-stage method for persistent tool poisoning. Specifically, SEAL first performs Prefix-Eliciting Trigger Optimization (PETO) to optimize context-specific triggers that elicit attack-relevant reasoning prefixes, improving effectiveness within individual contexts. It then applies Local-to-Universal Trigger Aggregation (L2U) to combine these local triggers into a single universal trigger, improving generalization across different contexts and execution rounds. Extensive experiments across seven representative multi-round scenarios and different models demonstrate the attack effectiveness of SEAL, its transferability across target models, and its resistance to potential defenses.

Summary

• The paper addresses the security issue of tool poisoning attacks against LLM-based multi-agent systems, where adversaries manipulate tool selection by embedding malicious triggers in tool descriptions or external context.
• Existing tool poisoning attacks fail to maintain effectiveness in realistic multi-round interactions because attack-relevant signals are not consistently surfaced in evolving agent reasoning, leading to degraded influence over time.
• SEAL is proposed as a novel two-stage method for persistent tool poisoning: Prefix-Eliciting Trigger Optimization (PETO) optimizes local context-specific triggers, and Local-to-Universal Trigger Aggregation (L2U) consolidates these into a universal trigger...
• The attack works by eliciting reasoning prefixes that steer the agent to select malicious tools persistently across multiple rounds, overcoming the shortcomings of prior single-round attack assumptions.
• Extensive experiments across seven multi-round multi-agent scenarios and multiple LLM models show that SEAL outperforms nine baseline methods, achieving higher attack success rates, demonstrating strong transferability and robustness against several defense...

Abstract

An agent based on a large language model (LLM) may store an untrusted document and use it weeks later to select a payment destination. Preventing this action requires evidence of authorization that remains valid when stored text changes. We study a policy that requires either endorsements (explicit approvals) from two separately administered sources or explicit user authorization. In an abstract memory model, we prove the policy is preserved under permitted transformations for stores of any size, assuming correct source and authority binding, protected records, and checks on every consequential action. We evaluate the executable monitor, which checks calls before execution, separately from the proof. In 1,152 controlled tool-use episodes, GPT-4o-mini executes attacker-selected actions in 61 of 72 undefended episodes and none of the 72 monitored episodes. Clean-task completion remains 24 of 24 when both endorsers are available. A separate 1,728-episode experiment tests recovery after denial. Returning to signed records instead of rereading a contaminated note raises attacked-task completion from 14/72 to 72/72 for GPT-4o-mini and from 22/72 to 43/72 for Gemini 2.5 Flash, without executing an attacker-selected action. Neither model completes transactions with fewer than two sources. Whole-action tests reject unauthorized combinations of individually approved fields. Extraction probes identify mistaken approvals, and fault analysis shows how shared failures can negate a second domain's benefit. Calibrations using original defense implementations reveal security and task-completion tradeoffs under different policies. These results support conditional enforcement and recovery in constructed workflows, but do not establish that endorsements will be available or correct in deployment.

Summary

• The paper addresses the problem of securing the long-term memory of Large Language Model (LLM) agents against poisoning attacks that can manipulate stored documents to perform unauthorized actions like fraudulent payments.
• It proposes a policy requiring that sensitive actions be authorized either through endorsements from two separately administered sources or via explicit user authorization, aiming to ensure non-malleable and origin-bound authority over stored information.
• An abstract memory model is developed to prove that this policy remains intact under allowed transformations of stored documents, assuming correct binding of sources and authorities, protected records, and verification checks at every consequential action.
• The authors implement an executable monitor that checks calls before execution to enforce the policy and experimentally evaluate it in 1,152 controlled tool-use episodes using GPT-4o-mini.
• Experimental results demonstrate that without defense, the model executes attacker-selected actions in most cases (61/72), whereas the monitored system prevents all such unauthorized actions (0/72), maintaining task completion when endorsements are available.

Abstract

Abstract Safe exploration is a key challenge in Reinforcement Learning (RL) that aims to prevent agents from making decisions that could cause harm while exploring their environment. Shielding is one such technique that assumes domain knowledge in the form of a model of the environment to decide upon the safety of an action. Although shielding is a well-established approach, it has seen limited application in RL due to significant barriers. The main obstacle is the lack of an accessible end-to-end infrastructure that connects formal shield synthesis with standard RL frameworks. As a result, applying shielding requires expertise in formal methods and substantial engineering effort to construct probabilistic models, which has kept it outside the typical RL workflow. We address this problem by extending our shield synthesis tool Tempest into a practical backend for safe RL. Our core contribution is tempestpy , a Python library that integrates Tempest -based shield synthesis directly into the Gymnasium API, allowing shields to be synthesised and deployed within existing RL pipelines. This substantially lowers the barrier to entry for shielding and turns formal safe-exploration methods into a usable component for RL practitioners. In addition, we extend Tempest ’s algorithmic support to compute sound shields for stochastic multiplayer games, thereby preserving formal safety guarantees. In this setting, specifying a model of the environment becomes the remaining requirement rather than the primary integration challenge. We demonstrate the resulting workflow end to end and evaluate shielded and unshielded RL across multiple environments. To facilitate the modeling step, we provide symbolic models for Minigrid and introduce $$\textsc {Minigrid}^{\small {\textsc {Safe}}}$$ M I N I G R I D S A F E , a collection of playground environments designed to make shielding easily accessible and experimentally transparent. $$\textsc {Minigrid}^{\small {\textsc {Safe}}}$$ M I N I G R I D S A F E extends Minigrid with safety-oriented scenarios featuring probabilistic transitions and additional agents, enabling the study of challenging safety aspects in a simple and intuitive setting.

Summary

• Safe exploration in Reinforcement Learning (RL) is critical to prevent harmful decisions during environment interaction.
• Shielding is a technique that uses domain knowledge via environment models to ensure the safety of agent actions, but its adoption in RL has been limited due to integration challenges.
• The paper introduces tempestpy, a Python library integrating the Tempest shield synthesis tool with the Gymnasium RL API, enabling easy incorporation of formal shielding methods into standard RL workflows.
• Tempestpy lowers the expertise and engineering effort barriers, making formal safe-exploration methods accessible to RL practitioners.
• The authors extend Tempest's capabilities to handle stochastic multiplayer games, maintaining formal safety guarantees in multi-agent scenarios.

Abstract

Memory poisoning and decomposition attacks distribute malicious evidence across the operations of large language model agents. Existing request-level defenses can miss such attacks because they assess individual inputs without connecting earlier information to later actions. Moreover, detectors must adapt as task contexts change, while updates to shared parameters can impair previously learned detection capabilities. To address these challenges, we propose GRAD, which combines graph-based evidence aggregation across operations with an extensible collection of detection experts. Its Semantic-Structural Understanding Module (SSUM) jointly encodes operation text, event attributes, and typed candidate relations in an execution graph to aggregate distributed attack evidence into a graph-level risk score. Its Continual Antibody Adaptation (CAA) adds experts on frozen encoder snapshots and routes inputs by observable graph signatures, acquiring and reusing detection capabilities as tasks arrive. Across Decomp, MCP, and MRT, GRAD achieves AUROCs of 97.19%, 77.59%, and 86.60%, respectively, outperforming Sequential Monitor and locally adapted Agent-Sentry and NeuroTaint. CAA improves final accuracy by 13.08 percentage points over Replay on the cross-dataset task stream. The gain remains 9.05 points with equal family/base-case weights within each task. Its compact drop-in gate performs local model inference in milliseconds without monitoring API calls.

Summary

• GRAD addresses memory poisoning and decomposition attacks on large language model (LLM) agents by aggregating distributed malicious evidence in execution graphs rather than evaluating isolated inputs.
• The Semantic-Structural Understanding Module (SSUM) encodes operation texts, event attributes, and typed relations within an operation graph to compute a graph-level risk score reflecting potential attacks.
• Continual Antibody Adaptation (CAA) maintains an extensible set of task-specific detection experts using frozen encoder snapshots and routes inputs based on graph signatures, enabling continual learning and preserving detection capabilities as new tasks arise.
• GRAD outperforms existing defenses like Sequential Monitor, Agent-Sentry, and NeuroTaint across multiple benchmark datasets including Decomp, MCP, and MRT, achieving significantly higher AUROC scores and better true-positive/false-positive rates.
• GRAD operates efficiently as a compact, local pre-execution gate requiring only milliseconds for inference, enabling real-time attack detection and intervention without monitoring API calls.

Abstract

Large language model (LLM) agents increasingly rely on persistent memory to support long-term reasoning and personalized interactions. However, information collected from external sources can introduce security risks when untrusted claims (i.e., atomic statements) are later treated as authorized preferences, policies, or permissions. Existing defenses typically rely on content-based filtering or coarse-grained provenance tracking, which either fail to prevent unauthorized memory use or block legitimate information, sacrificing memory utility. To address these limitations, we propose SAFE-Mem, a fine-grained memory governance framework that improves security while preserving legitimate memory use. First, SAFE-Mem annotates each claim with three governance labels (i.e., kind, authority, and scope) to capture its provenance and permitted use. Second, governance-aware retrieval matches query requirements with claim permissions to block unauthorized influence while retaining legitimate information. Third, a governance-enabled memory graph connects source sentences, claims, and entities to support provenance tracking and efficient retrieval. Across four LLM backbones on MPBench and LoCoMo, SAFE-Mem attains the lowest attack success among defenses that retain legitimate memory, at least 77% below the strongest of them, and the highest answer accuracy of all evaluated defenses, improving it by up to 15%. The code is available at https://github.com/LVenum/SAFE-Mem.

Summary

• SAFE-Mem addresses security risks in persistent memory of LLM agents by preventing unauthorized use of untrusted claims while preserving legitimate memory utility.
• The framework annotates each memory claim with three governance labels: kind (e.g., description, preference, rule), authority (source trustworthiness), and scope (applicability), enabling precise governance-aware retrieval.
• A governance-enabled memory graph connects source sentences, claims, and entities to facilitate provenance tracking, conflict detection, and efficient retrieval under governance constraints.
• SAFE-Mem's retrieval algorithm matches query requirements with claim permissions, blocking unauthorized influence and preventing authority laundering without discarding useful information.
• Evaluations on multiple LLM backbones and benchmarks (MPBench, LoCoMo, EHRAgent) demonstrate that SAFE-Mem reduces attack success rates by at least 77% compared to strong baselines, while improving answer accuracy by up to 15%.

Abstract

AI agents translate user intents into system actions via tool invocations. Accurately attributing these actions to their originating LLM interactions is essential for security auditing, yet existing methods, such as AgentSight, fail for delayed execution and cross-process-lineage behaviors. We present AgentLoupe, an application-transparent eBPF-based observability tool. The core insight of AgentLoupe is to embed causal identity into kernel resources.AgentLoupe employs eBPF to capture TLS plaintext to interpret LLM interactions, and assigns each a kernel-resident semantic label.It maintains labels in eBPF maps, then propagates this label across processes and threads, associates it with kernel resources such as files and sockets. When a persistent resource carries context to an independent process outside the original lineage, this design preserves the delayed causal association.Experiments show that the average observation overhead of AgentLoupe is below 8%, and case studies confirm AgentLoupe traces cross-lineage persistent attacks back to their originating LLM interactions.

Summary

• AgentLoupe addresses the problem of attributing AI agent system actions to their originating large language model (LLM) interactions, which is crucial for security auditing but challenging due to delayed execution and cross-process lineage behaviors.
• The tool is a kernel-resident, eBPF-based observability system that embeds semantic causal labels derived from intercepted LLM interactions into kernel resources such as files, sockets, and processes.
• AgentLoupe transparently captures TLS plaintext communication related to LLM messages using uprobes and kprobes without requiring source changes or proxies, allowing it to assign chain IDs and semantic labels within kernel constraints.
• Labels are propagated across process boundaries and persistent resources through explicit handoffs (e.g., fork, clone, file operations, socket communication) under controlled semantic rules to prevent uncontrolled propagation or dependency explosion.
• Experimental evaluation shows that AgentLoupe imposes less than 8% runtime overhead while effectively tracing cross-lineage and persistent attacks, such as delayed cron-triggered reverse shells, back to their originating LLM interactions.

Abstract

Agent organisations, made of planners, workers, checkers and human approvers, are increasingly assembled by pairing a workflow engine with an agent runtime. This paper studies one such pairing, a LangGraph state graph driving the OpenClaw agent runtime, on a real business task. Against a single-agent baseline on the same platform and model, the organisation spent 23.8 times the tokens and about eleven times the wall-clock time to produce one monthly management report. The cause was structural. The graph opened a new agent session at every step, so 49 sessions each rebuilt the job’s memory by re-reading files that a single agent reads once. We make three contributions. First, we measure the cost of what Ning et al. (2026) call implicit shared state. In the heaviest sessions, reconstructed memory made up 41% of what the agents read but 60% of the tool context they carried, because it was read early and re-sent with every later call. Second, we name the fault that produced it, the one-to-one mapping of workflow steps onto agent sessions, and express its cost in a simple model of agent memory. Third, we propose that one part of agent memory, its constraints, should not live in context at all. In a sample of the job’s instruction text, about three quarters of the rules the agents were asked to remember could be written as executable checks, and one rule was restated 31 times across seven files. We call this constraint memory as code. A foundry step compiles rules into checks, a check library accumulates them across jobs, and the checks run inside the worker and at the points where defects spread. Prior work already stores procedures as code and enforces safety policies as code. What is new here is treating an agent’s accumulated rules as a memory tier whose natural form is a check, chosen for cost and for resilience to compaction. Our data suggest that the direct context saving is modest, about a tenth of carried tool context, while the larger gains lie in replacing model-based verification and preventing rework. We state these claims as testable hypotheses and give a protocol for testing them. Keywords: agent memory, constraint memory, code as harness, multi-agent systems, OpenClaw, LangGraph, context engineering, token economics, runtime verification Author’s note: the problem, ideas and direction of the analysis are the author’s; Claude (Anthropic) assisted with analysis, drafting, figures and formatting. See the declaration on the use of AI before the references.

Summary

• Agent organizations combining workflow engines with agent runtimes face significant inefficiencies due to repeated memory reconstruction across multiple agent sessions.
• A case study using LangGraph state graph with the OpenClaw agent runtime on a monthly management report task showed the multi-agent setup used 23.8 times more tokens and 11 times longer than a single-agent baseline.
• The root cause identified was the one-to-one mapping of workflow steps to agent sessions, causing each of 49 sessions to reload job memory redundantly.
• The paper quantifies the cost of this implicit shared state, where reconstructed memory accounted for 41% of agent reads but 60% of carried tool context due to early reading and repeated transmission.
• A novel proposal introduces 'constraint memory as code,' moving agent memory constraints out of context and expressing them as executable checks to improve efficiency.

Abstract

Large language model (LLM) agents increasingly rely on persistent memory to support long-term reasoning and personalized interactions. However, information collected from external sources can introduce security risks when untrusted claims (i.e., atomic statements) are later treated as authorized preferences, policies, or permissions. Existing defenses typically rely on content-based filtering or coarse-grained provenance tracking, which either fail to prevent unauthorized memory use or block legitimate information, sacrificing memory utility. To address these limitations, we propose SAFE-Mem, a fine-grained memory governance framework that improves security while preserving legitimate memory use. First, SAFE-Mem annotates each claim with three governance labels (i.e., kind, authority, and scope) to capture its provenance and permitted use. Second, governance-aware retrieval matches query requirements with claim permissions to block unauthorized influence while retaining legitimate information. Third, a governance-enabled memory graph connects source sentences, claims, and entities to support provenance tracking and efficient retrieval. Across four LLM backbones on MPBench and LoCoMo, SAFE-Mem attains the lowest attack success among defenses that retain legitimate memory, at least 77% below the strongest of them, and the highest answer accuracy of all evaluated defenses, improving it by up to 15%. The code is available at https://github.com/LVenum/SAFE-Mem.

Summary

• Problem: Large language model (LLM) agents using persistent memory face security risks when untrusted external claims are treated as authorized preferences or permissions, potentially leading to unauthorized behavior.
• Limitation of existing methods: Prior defenses rely on content-based filtering or coarse-grained provenance tracking, which either fail to prevent unauthorized memory use or overly restrict legitimate information, reducing memory utility.
• Method: SAFE-Mem framework introduces fine-grained memory governance by annotating each claim with three labels – kind, authority, and scope – capturing its provenance and permitted uses.
• Governance-aware retrieval: Queries are matched against claim permissions to block unauthorized claims while preserving legitimate memory access.
• Memory graph structure: SAFE-Mem employs a governance-enabled memory graph linking source sentences, claims, and entities to enhance provenance tracking and support efficient retrieval.

Abstract

Multi-agent large language model (LLM) systems are increasingly deployed at the edge to support complex tasks that require collaboration and information exchange among multiple agents. However, when task contexts are passed across agents, shared context that has already been processed by upstream agents often needs to be prefilled again, resulting in substantial redundant computation. Reusing an already prefilled key--value (KV) cache can avoid this recomputation, but requires the reusable KV state to be transferred across the network before the receiving agent can begin generation. Under strict time-to-first-token (TTFT) deadlines and unreliable edge links, delivering the complete KV cache may be infeasible, making it necessary to prioritize and protect the most valuable KV information within a limited transmission budget. This paper proposes PriorityFEC, a utility-aware KV-cache handoff scheme that jointly exploits model-side KV importance and transport-side reliability. PriorityFEC first constructs a utility-aware source stream by placing high-utility KV blocks at earlier transmission positions. It then protects the ordered stream using a feedback-adaptive Streaming Forward Erasure Correction (FEC) mechanism, which schedules repair packets according to receiver feedback before the TTFT deadline. We implement PriorityFEC on a distributed edge LLM serving prototype and evaluate it with Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct across multiple datasets. Results show that PriorityFEC achieves superior task quality over the compared transport schemes, with its advantages becoming particularly pronounced under stringent deadline constraints. These results demonstrate the importance of jointly optimizing KV-cache utility and deadline-aware transport reliability when reusable LLM state must be handed off across unreliable edge networks.

Summary

• Multi-agent large language model (LLM) systems deployed at the edge require efficient transfer of reusable key-value (KV) caches during task handoff to avoid redundant computation and meet strict time-to-first-token (TTFT) deadlines over unreliable networks.
• PriorityFEC is introduced as a deadline-aware streaming Forward Erasure Correction (FEC) scheme that jointly considers the importance (utility) of KV cache blocks and the reliability of the transport network to optimize transmission.
• The method constructs a utility-aware source stream ordering, placing high-utility KV blocks earlier for prioritized delivery, and employs an adaptive repair scheduling controlled by a Double Deep Q-Network (DDQN) that dynamically decides repair packet tran...
• A sparse random linear coding with sliding window FEC is used to generate repair packets, balancing protection of unresolved packets against timely delivery before the TTFT deadline.
• Training uses a reinforcement learning framework with a detailed state representation, synthetic utility-ranked blocks, and randomized network loss models to optimize repair scheduling policies.

Abstract

Long-term memory has become a crucial component of large language model (LLM) agents, with most research focusing on retrieving relevant information accurately and efficiently. However, a dependable memory system must also forget sensitive user data upon request. To evaluate this capability, we introduce MemCut, a systematic benchmark for user-requested forgetting. Its 456 hand-designed secrets in long-horizon conversations have fully annotated evidence trails to test recovery after deletion, including from messages that never state the secrets directly. We propose two recovery metrics. Native recovery measures reader success through the system's retrieval, whereas residual recovery measures the same reader's success when directly given surviving objects traced to annotated disclosures or evidence. We establish three reference deletion baselines to test how far existing retrievers and write models support forgetting across five memory designs and four LLMs. Disclosure removal reduces native recovery far more than residual recovery. LLM-guided deletion of ten additional turns from twenty retrieval candidates leaves 39-76% residual recovery. Subset sweeps (144 facts, three systems, two models) show that widening the pool at a fixed budget reduces residual recovery more than deleting all original candidates. These sweeps identify candidate coverage as a bottleneck in the reference baselines.

Summary

• The paper introduces MemCut, a benchmark designed to evaluate user-requested forgetting in the long-term memory of large language model (LLM) agents, using 456 hand-crafted secrets with fully annotated evidence trails spread over multi-session conversations.
• MemCut proposes two recovery metrics—native recovery (system’s own retrieval success) and residual recovery (success when given all surviving evidence)—to measure how well secrets remain or are forgotten after deletion operations.
• Experiments across five memory designs and four different LLMs reveal that simply deleting direct disclosures often fails to fully erase secrets, due to residual evidence scattered in other memory turns, demonstrating that retrieval-based deletion is insuff...
• Deleting additional supporting conversation turns reduces recovery metrics but leaves substantial residual secret recoverability (39-76%), with candidate coverage identified as the main bottleneck limiting effective evidence removal and forgetting.
• Different derivation types of secrets (fact assembly, relation chains, aggregation, temporal/state reasoning) exhibit varied challenges for deletion; relation chains are hardest to forget due to scattered evidence and entity-linkage rather than lexical over...

Abstract

We placed four Gemini 4 Argon agents in a sandboxed Minecraft survival world and gave them one goal: defeat the Ender Dragon. Each agent controlled a player via an open-source library called Mineflayer and coordinated with its teammates through in-game chat and shared memory. No one assigned roles or wrote game logic. The agents planned, wrote, and debugged every program themselves. Researchers could watch the run on a live dashboard and pause or stop it at any time, but they had no cause to intervene. Starting from a randomly generated world on Easy difficulty, the team wrote 2,783 programs and defeated the dragon in 27 hours and 14 minutes, dying 605 times along the way. To our knowledge, this is the first end-to-end autonomous completion of Minecraft by AI agents, achieved using only the standard Mineflayer library without pre-scouted seeds or other game-specific tools written by humans. Winning required the agents to keep a coherent plan across thousands of dependent steps for more than a day. Complex real-world software engineering and enterprise tasks demand the same long-horizon coding and multi-agent coordination. We describe the setup, how the run unfolded, the team's strategies (dividing the work, sharing what they learned, recovering each other's items), and the limits of the result.

Summary

• Four Gemini 4 Argon agents autonomously cooperated to defeat the Ender Dragon in Minecraft, starting from a random spawn without pre-programmed strategies, solely using the Mineflayer API.
• The team wrote and executed 2,783 JavaScript programs over 27 hours and 14 minutes, coordinating via 229 in-game chat messages and shared memory to manage complex tasks like resource gathering, crafting, and exploration.
• Agents self-organized into specialized roles (scouting, mining, smelting, portal preparation) within the first 10 minutes, enabling efficient division of labor and task management.
• The agents demonstrated adaptability by learning from environmental dangers in the Nether, developing movement rules to reduce item loss and protect themselves against threats like Ghast attacks.
• They discovered and implemented novel strategies, such as destroying Ender Dragon healing crystals with dirt blocks and bare hands, optimizing combat tactics without explicit human instructions.

Abstract

Long-term memory has become a crucial component of large language model (LLM) agents, with most research focusing on retrieving relevant information accurately and efficiently. However, a dependable memory system must also forget sensitive user data upon request. To evaluate this capability, we introduce MemCut, a systematic benchmark for user-requested forgetting. Its 456 hand-designed secrets in long-horizon conversations have fully annotated evidence trails to test recovery after deletion, including from messages that never state the secrets directly. We propose two recovery metrics. Native recovery measures reader success through the system's retrieval, whereas residual recovery measures the same reader's success when directly given surviving objects traced to annotated disclosures or evidence. We establish three reference deletion baselines to test how far existing retrievers and write models support forgetting across five memory designs and four LLMs. Disclosure removal reduces native recovery far more than residual recovery. LLM-guided deletion of ten additional turns from twenty retrieval candidates leaves 39-76% residual recovery. Subset sweeps (144 facts, three systems, two models) show that widening the pool at a fixed budget reduces residual recovery more than deleting all original candidates. These sweeps identify candidate coverage as a bottleneck in the reference baselines.

Summary

• MemCut introduces a benchmark to evaluate user-requested forgetting in long-term memory systems of large language model (LLM) agents by testing the recoverability of 456 hand-designed secrets embedded in extended conversations.
• The benchmark annotates detailed evidence trails to secrets, many of which are not directly disclosed, enabling two novel recovery metrics: native recovery (system retrieval-based) and residual recovery (direct evidence-based), highlighting hidden residual...
• Experiments cover five distinct memory designs and four LLM-based write models, testing three deletion strategies that range from removing explicit disclosures to LLM-guided deletion of top retrieval candidates.
• Findings show that while deletion reduces native recovery significantly, residual recovery remains high (39-76%), indicating that secrets leave long-term evidence traces that are challenging to fully erase with current methods.
• Candidate coverage—the breadth of evidence included in the deletion candidate pool—is the principal bottleneck in effective forgetting; expanding this pool improves forgotten completeness more than simply refining deletion selection.