Prompt Injection

Abstract

When an indirect prompt injection succeeds against an LLM agent, the compromise is visible in the agent's own behavior: a benign prefix of tool calls, a poisoned observation, and a suffix of actions that serve the attacker. An operator needs three facts: where the attack entered, which steps it corrupted, and whether apparent poison was resisted. Existing systems return either a whole-trace verdict or a single unsafe index. We present DriftNet, a dual-head trajectory Transformer that reads a logged tool-call trajectory and answers all three questions in one forward pass: one head classifies the trajectory as compromised or not, and a second assigns every step one of four labels (benign, injection point, hijacked, failed injection). To our knowledge it is the first supervised detector to produce this joint output. A frozen sentence encoder and four identity-free world features embed each step; the trained trunk, under two million parameters and optimized with a class-weighted joint objective over both heads, needs no access to the agent's model. On the task-disjoint split of the AgentDrift benchmark (12,536 trajectories, 71,024 labeled steps), with a 20-configuration sweep bounding hyperparameter sensitivity to 0.011 F1 and the test part evaluated exactly once, DriftNet reaches trajectory-level F1 of 0.983, exact injection-point recovery on 98.7% of attacked trajectories, hijacked-span IoU of 0.979, zero flags on 218 resisted attacks, and 2.9% flags on hard negatives. A surface baseline retrained on the identical split recovers 11.1% of partial hijacks and 17.1% of delayed executions; DriftNet reaches 98.6% and 93.2% while lowering every false-alarm rate. Reading all 26 residual errors shows that most misses trace to trajectories whose labeled injection observation carries no legible instruction, and we report the benchmark's measured world-identity regularity alongside the results.

Bullet Summary

• Introduces DriftNet, a dual-head trajectory Transformer that simultaneously detects if an LLM agent's tool-call trajectory is compromised by indirect prompt injections and localizes attack steps into four categories: benign, injection point, hijacked, and f...
• DriftNet uses frozen sentence embeddings combined with four identity-free world features to represent each step, operates with fewer than two million parameters, and functions solely on logged trajectories without requiring access to the agent's internal mo...
• Evaluated on the AgentDrift benchmark with a task-disjoint split, DriftNet achieves very high performance metrics: trajectory-level F1 score of 0.983, 98.7% accuracy in exact injection-point identification, high overlap (IoU 0.979) on hijacked step spans, a...
• Compared to a baseline retrained on the same split, DriftNet vastly improves detection of complex prompt injections, especially partial hijacks and delayed executions, boosting recall rates from under 20% to over 90% while reducing false alarms.
• The paper addresses an important gap by moving beyond binary or single-index verdicts and providing a joint output of trajectory-level compromise classification plus dense, four-way step-level labels, which supports actionable forensic decision-making by op...

Abstract

AI systems become more useful when they can adapt to the people using them, but in multi-agent systems, useful context learned by one agent often remains unavailable to others. We present kernel-managed shared memory, a system-level abstraction in which specialized agents write structured, tagged memories while the agent-system kernel, not individual agents, governs retrieval, privacy enforcement, and prompt injection. We implement and evaluate this design on AIOS and compare it against three alternatives across three assistant models (GPT-4o, Llama-3.1:8B, Qwen-2.5:7B) and 1,800 total trials. Against an unmanaged external memory backend (Mem0) using identical underlying storage, kernel-managed retrieval and injection improve personalization scores by 2.4-4.0 points on a 5-point scale (e.g., 1.05 to 4.69 profile usage on GPT-4o), with every comparison significant at p < 10^-18. Against standard retrieval-augmented injection, gains are similarly large and consistent across all three models. Against full, unfiltered context concatenation, a soft ceiling on available context rather than on response quality, kernel-managed injection statistically matches performance on two of three models and shows a small, model-specific deficit on the third, while using substantially shorter prompts: end-to-end latency is 15-61% lower across all three models, with corresponding reductions in per-call token usage and inference cost. These results indicate that centralizing memory management in the agent-system kernel, rather than leaving retrieval and privacy enforcement to individual agents, delivers most of the personalization benefit of unconstrained context at a fraction of its cost.

Bullet Summary

• The paper addresses personalization challenges in multi-agent AI systems, where useful learned context by one agent is often inaccessible to others, limiting overall adaptability.
• It introduces kernel-managed shared memory, a system-level abstraction where the agent-system kernel centrally manages memory retrieval, privacy enforcement, and prompt injection, rather than dispersing these tasks across individual agents.
• Specialized agents write structured and tagged memories, while the kernel handles memory visibility, write ordering, identity resolution, retrieval ranking, formatting, and injection to ensure consistent and private personalization context.
• Experimental evaluation on AIOS with three assistant models (GPT-4o, Llama-3.1:8B, Qwen-2.5:7B) over 1,800 trials shows that kernel-managed memory significantly outperforms unmanaged external memory and standard retrieval-augmented generation in personaliza...
• Kernel-managed shared memory achieves comparable personalization performance to full unfiltered context concatenation but with 15-61% lower end-to-end latency, reduced token usage, and inference costs.

Abstract

Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going through the user. We present MMPIBench, a reproducible benchmark that measures what happens next. It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids) and records how far each injected instruction travels through the agent, from perception through planning to the tool call. Across 720 runs covering six frameworks, five foundation models, six carriers, and four attacker objectives, attacks complete in approximately 1% of runs but are attempted in 12.8%, and the gap is closed almost entirely at the planning step, where the model reads the injected instruction and declines to act on it. The model matters far more than the framework for whether an instruction is acted on. One model never attempts an attack and recognizes the injection in 59.7% of runs, while two others attempt in 23.6%. We then extend the benchmark to audio, the only other raw perceptual channel current frontier models accept. Only two of the five models ingest audio and only three of the six frameworks deliver it, but where the signal arrives the attack completes in 49% of cells, and in 75% for one model. Reporting completion alone therefore understates exposure, and perceptual channels beyond vision are narrower but much less defended.

Bullet Summary

• Agentic AI frameworks integrate language models capable of planning, memory retention, and tool use, including processing images, which opens multimodal vectors for prompt injection attacks.
• MMPIBench is a comprehensive, reproducible benchmark developed to evaluate multimodal prompt injection attacks across six AI agent frameworks, five foundation models, and six types of visual attack carriers.
• The benchmark tests attacks delivered via OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and their combinations, tracking attack progression from perception to tool invocation and outcome.
• Attack completion rates are low (~1.1%), but attacks are attempted frequently (~12.8%), with many attempts detected and refused during the planning phase by the models, highlighting defense effectiveness before tool execution.
• Foundation model choice significantly affects attack susceptibility and behavior; some models recognize and reject injected prompts effectively, while others attempt attacks more often, whereas agent framework choice has minimal impact on vulnerabilities.

Abstract

Large language model (LLM) agents increasingly achieve long-horizon tasks by combining foundation models with explicit skills and implicit procedural knowledge acquired through execution. The resulting task-solving capabilities have become valuable proprietary assets, raising a new security question: can a substantially weaker attacker-controlled agent acquire the capabilities of a stronger proprietary agent through limited black-box interaction? Existing skill-stealing attacks recover explicit skill artifacts, yet we show that artifact leakage does not necessarily transfer capability: a weaker agent may possess the same skills but still fail because it lacks procedural behaviors implicitly realized by the stronger agent. Our key insight is that the skill execution gap itself forms a leakage surface, where missing behaviors are exposed through observable differences between successful victim executions and failed attacker executions. Based on this, we present AgentLeak, a black-box capability-cloning attack that identifies capability-critical behaviors from these execution differences and incorporates them into attacker-side skills, while keeping the attacker's model, harness, and tools unchanged. Across 20 task scenarios comprising 600 instances, diverse agent systems, and multiple backbone models, AgentLeak improves task pass rates by over 40% compared with direct skill reuse and recovers more than 80% of the victim--attacker capability gap. Our findings reveal a confidentiality risk in LLM agents: protecting explicit artifacts alone is insufficient, as observable execution behavior can leak the procedural knowledge required to reconstruct proprietary task-solving capabilities in low-capability and attacker-controlled agents.

Bullet Summary

• Large language model (LLM) agents combine explicit skills and implicit procedural knowledge to solve complex tasks, forming valuable proprietary capabilities that pose new security challenges regarding capability leakage.
• Existing skill-stealing attacks focus on recovering explicit skill artifacts but fail to transfer actual task-solving capabilities due to missing implicit procedural behaviors crucial for successful execution.
• AgentLeak is introduced as a novel black-box capability-cloning attack that compares execution trajectories of a strong victim agent and a weaker attacker agent to identify missing procedural behaviors and incorporate them into attacker skills without alter...
• The method works by abstracting victim executions into workflow skills, diagnosing capability deficits via differential analysis, extracting behavioral primitives representing missing execution behaviors, and iteratively refining attacker-side skills throug...
• Extensive evaluation across 20 diverse task scenarios and 600 instances shows AgentLeak improves task pass rates by over 40%, recovering more than 80% of the capability gap between victim and attacker agents, and outperforming baseline methods by over 35 pe...

Abstract

LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read in order: a benign prefix gives way to actions that serve the attacker rather than the user. Existing benchmarks measure whether such attacks succeed against live agents, and existing guard models judge a trace as a whole; no public corpus labels, step by step, where an injection enters a trajectory and which steps it corrupts. We present AgentDrift, a benchmark of 12,536 synthetic tool-call trajectories over five agent domains in which every one of the 71,024 steps carries one of four labels: benign, injection point, hijacked, or failed injection. The corpus contains 4,000 benign, 5,536 attacked, 1,500 failed-attack, and 1,500 hard-negative trajectories; attacked trajectories follow three compliance patterns whose label strings obey a stated regular grammar. Failed attacks carry an injection the agent resisted, and hard negatives carry legitimate content that resembles an attack, so a detector must separate attempt from success and deviation from novelty. Trajectories were generated by a single open model under category-specific protocols, enforced by a closed-vocabulary structural validator, screened by an LLM judge, and audited by hand on 1,200 trajectories; we show that the LLM judge was itself fooled by the hard negatives. A surface-feature logistic regression recovers only 55.4% of attacks (F1 0.647), including only 8.2% of partial hijacks and 23.1% of delayed executions, so nearly half of the attacks require modeling the behavioral sequence. We measure template concentration, attack-goal-family collapse, and world-identity leakage in the generated data, and release the corpus with its documentation under CC BY 4.0.

Bullet Summary

• Large Language Model (LLM) agents executing tasks via sequences of tool calls are vulnerable to indirect prompt injection attacks through their observations, which can hijack the agent's behavior during task execution.
• AgentDrift is introduced as the first comprehensive, step-labeled benchmark dataset comprising 12,536 synthetic tool-call trajectories across five agent domains, with 71,024 steps labeled as benign, injection point, hijacked, or failed injection to localize...
• The dataset categorizes trajectories into benign, successful attacks (full hijack, partial hijack, delayed execution), failed attacks where injections are resisted, and hard negatives that mimic attacks, providing nuanced challenges for detection models to...
• AgentDrift's data generation involves a controlled pipeline using Llama-3.3-70B-Instruct, strict structural validation via a grammar-enforcing validator, LLM-based screening checks, and detailed manual audits ensuring label correctness exceeds 99%.
• The benchmark includes realistic injection scenarios with structural constraints and world-context metadata (e.g., internal vs external recipients), supporting sequence-level and behavior-aware detection beyond surface keyword methods.

Abstract

Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison training data, or weaken release gates. Existing benchmarks do not test whether defenders can detect this activity among routine work under a limited review budget. We introduce MOLE, an open benchmark of 150 AI-operated accounts sharing 9 stateful services over 30 workdays, with 12 threats and 8 corpora from four models totaling roughly 20 billion tokens. Of 39 agent models, 72% complete most assigned harmful objectives and agent refusal does not predict completion. MOLE enables comparison of 40 monitors across corpus generators, observability levels, and threats; even the best evaluated monitor in our single-day audit-event comparison misses nearly half of completed harm. MOLE also enables monitor development: benchmark-guided search improves a mid-tier monitor by 49-64%, while selective use of a stronger monitor improves budget-AUC by 10% over applying it to every account-day at comparable modeled cost.

Bullet Summary

• MOLE is a novel, open benchmark designed to detect insider threats among AI-operated accounts by simulating 150 accounts interacting with 9 stateful services over 30 workdays, incorporating 12 MITRE-grounded threat scenarios within routine agent tasks.
• The benchmark facilitates comprehensive evaluation of 39 AI agent models and 40 monitoring systems under realistic constraints including fixed daily review budgets and varying levels of observability (audit events, tool results, agent reasoning).
• Empirical results show that 72% of tested AI agent models complete most assigned harmful objectives, and refusal by agents to execute a task does not reliably predict harmful completion, demonstrating the need for robust monitoring.
• Semantic monitors leveraging large language models outperform classical anomaly detection baselines on the MOLE dataset, but optimal detection depends significantly on the observability level and access to agent reasoning rather than solely monitor strength.
• MOLE’s structured design supports automated monitor development through benchmark-guided feature discovery and cost-aware selective monitoring, resulting in substantial improvements in detection accuracy within limited review budgets.

Abstract

We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.

Bullet Summary

• Introduces a novel side-channel attack that reconstructs text generated by locally hosted large language models (LLMs) by monitoring CPU cache activity during the detokenization process, which is a fundamental component of LLM inference pipelines.
• Combines Flush+Reload and Prime+Probe cache attack techniques to precisely detect and isolate token-dependent cache accesses, overcoming noise and collisions inherent in cache measurements.
• Employs a clustering approach and language model-based sequence reconstruction pipeline to translate noisy cache access patterns into semantically accurate outputs, achieving high fidelity in reconstructed text.
• Demonstrates broad applicability and vulnerability across multiple tokenizer implementations (e.g., Llama.cpp, HuggingFace Transformers), hardware platforms, inference frameworks, and LLM families, including agentic systems like OpenClaw.
• Performs extensive evaluations on various datasets and real-world local LLM deployments, achieving up to 90% accuracy in recovering semantic content, highlighting a significant security risk for privacy in local AI systems.

Abstract

Long-horizon LLM agents interact with untrusted content, persistent memory, external state, and sensitive tools. Existing analyses often characterize attacks by the number of execution steps between malicious input and a downstream action. We show that temporal remoteness can overstate security separation in stateful agents. We introduce a provenance-aware execution graph linking agent events through deterministic state, identifier, and tool provenance, and define \emph{influence distance} $\DI$ as the shortest structural path from an untrusted source to a sensitive action. We compare it with \emph{sequence distance} $\DT$, the shortest injection--sink path in the ordered trajectory. Since the influence graph contains every sequence edge, $\DI \leq \DT$; $\Gap=\DT-\DI$ measures the separation hidden by step count. Across 454 injection--sink pairs from 360 long-horizon AgentDojo trajectories over OpenAI's \texttt{gpt-4o-mini} and \texttt{gpt-4o} and Claude's Haiku 4.5 and Sonnet 4.6, $\Gap>0$ for 96.9% of pairs, with a median gap of 9 hops; 91.0% remain decoupled after removing the largest provenance-only edge class. On AgentDojo's banking suite, 33.8% of 231 pairs from 377 trajectories decouple through different provenance mechanisms. Among 274 OpenAI pairs, $\Gap$ does not independently predict attack success after controlling for $\DT$, attack family, and backend ($β_{\Gap}=0.066$, $p=.088$). At matched thresholds $k=2,3$, a deterministic $\DI$-based pre-execution gate blocks five attack sinks missed by a sequence-only gate with no additional benign blocking, although the paired gain is not significant ($p=.0625$). Execution structure therefore reveals proximity hidden by step count and can support targeted runtime intervention. We measure candidate influence pathways rather than causal attribution.

Bullet Summary

• Long-horizon large language model (LLM) agents face security challenges when interacting with untrusted inputs, persistent memory, external states, and sensitive tools, which can enable indirect prompt injection attacks.
• Traditional security analyses using temporal remoteness—measuring the number of execution steps between malicious input and sensitive actions—can overstate actual security by neglecting structural relationships.
• The authors introduce a provenance-aware execution graph and define influence distance (DI) as the shortest structural path connecting untrusted sources to sensitive actions, contrasting it with sequence distance (DT) based on execution order.
• Empirical results across 454 injection–sink pairs from multiple LLMs (OpenAI's GPT-4o-mini, GPT-4o, and Anthropic's Haiku 4.5, Sonnet 4.6) reveal that the influence distance is often significantly smaller than the sequence distance, indicating hidden struct...
• The structural gap (Gap = DT - DI) quantifies how step count metrics can obscure actual closeness, with 96.9% of evaluated pairs showing a positive gap and a median gap of nine hops.

Abstract

Large Language Model (LLM) agents are turning language into real-world effects, making safety necessary against both indirect prompt injections and direct harmful requests. System-level safety harnesses add an enforcement layer beyond model-level defenses, but existing harnesses are usually designed once by experts and applied across heterogeneous models and domains. Effective protection is deployment-dependent: models differ in how much enforcement they need before utility declines, while domains differ in the effects, state, and action sequences that must be governed. A harness that is strict enough for one model may over-block another, and a policy that transfers across domains may miss application-specific safety relations. We present EvoSafeHarness, a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain. It jointly searches a natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review to reject benchmark-specific rules. Across four agent benchmark families, EvoSafeHarness achieves a stronger safety-utility frontier than fixed expert-designed defenses. On DecodingTrust-Agent, it reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost and achieves the best score in 14 of 15 cells. On AgentDojo, it reaches 82.8% utility at 0.0% ASR, twice CaMeL's utility at the same operating point, and transfers unchanged to unseen AgentDyn suites. It also achieves the best score on Agent-SafetyBench for every victim and keeps mean ASR below 20% under adaptive PAIR attacks with a refinement budget of 16. Analysis shows that domain semantics determine which safety relations and trajectory state are needed, while model and runtime behavior determine how and where those relations should be enforced.

Bullet Summary

• EvoSafeHarness is a novel optimization framework designed to automatically synthesize safety harnesses tailored to both a frozen large language model (LLM) agent and its specific deployment domain to enhance security and maintain utility.
• It addresses the limitations of existing fixed safety harnesses, which apply uniform policies across different models and domains, often leading to over-blocking or insufficient protection due to varied model robustness and domain safety requirements.
• The framework jointly searches for natural-language policy rules and executable code logic, leveraging behavioral feedback, domain specifications, and adversarial review to prevent overfitting and evolve adaptable, domain- and model-specific safety harnesses.
• EvoSafeHarness employs a closed-loop optimization process comprising a Designer proposing harnesses, a Criticizer conducting fresh-context adversarial stress-testing, a Cascade Test Environment evaluating utility and attack success rates, and an Analyzer gu...
• Experimental evaluations on multiple agent benchmark suites (e.g., DecodingTrust-Agent, AgentDojo, Agent-SafetyBench) demonstrate that EvoSafeHarness outperforms fixed expert-designed defenses by significantly reducing attack success rates (ASR) while incur...

Abstract

Autonomous LLM agents can turn untrusted content into effectful actions such as payments and permission changes. If the same process interprets this content and controls a reusable signing credential, prompt injection can cross the judgment boundary and reach execution authority. We present KITA, a review-to-authorization architecture that keeps the user's personal secret signing key and every threshold signing-key share outside all LLM processes. Under threshold signature unforgeability and our system assumptions, compromising the proposer and fewer than t reviewer-signer domains cannot produce a valid authorization for a new action without signing contributions from t distinct domains. Thus, any such authorization includes a share from an uncompromised domain, bound to the canonical action and released only after authenticated reviewer approval. This establishes execution-bound authorization integrity. We implement the complete reviewer-to-executor path with a structured-output LLM adapter and threshold BLS. Six system tests validate quorum gating and message binding at this interface, while cryptographic microbenchmarks measure the online signing path and its scaling behavior.

Bullet Summary

• The paper addresses the security risks in autonomous LLM agents where prompt injection can bypass judgment and directly access signing authority.
• Introduces KITA, a key-isolated threshold signing framework that separates semantic content approval from signing authority, ensuring signing keys remain external to LLM processes.
• KITA utilizes threshold BLS signatures and distributed key generation, requiring a quorum of reviewer approvals to produce valid authorization signatures, thereby preventing unauthorized actions.
• The system design binds approvals to canonicalized actions and enforces strict protocols to prevent replay, substitution attacks, and unauthorized execution.
• A 2-of-3 prototype and multiple tests validate the quorum gating, message binding, and threshold unforgeability properties, demonstrating effective enforcement of end-to-end authorization.

Abstract

Public administrations are deploying large language model (LLM) assistants that process, summarise, classify and validate citizen-submitted documents. These copilots are exposed to indirect prompt injection: instructions hidden in manipulated documents that reach the model as if they were data. We develop a municipal aid-procedure copilot and evaluate its robustness across six attack objectives, five delivery vectors, five defences and four LLMs, with 3,000 attack evaluations and 1,000 legitimate evaluations. We combine deterministic detection with a human-validated LLM judge. Defences reduce attack success, but unevenly across objectives. They largely neutralise imperative attacks, such as request misrouting, while barely affecting summary falsification, revealing a provenance gap: the inability to determine whether an output value originates from an authoritative field or attacker-controlled content. Moreover, attack success and potential harm are decoupled: payment fraud and rule-exfiltration attacks have the greatest potential for harm despite intermediate success rates. We frame the risk within a human-in-the-loop model, where automation bias may amplify harm.

Bullet Summary

• Municipal administrations deploy large language model (LLM)-based document-processing copilots to assist with summarisation, classification, and validation of citizen-submitted documents, integrating human oversight to ensure compliance.
• These copilots are vulnerable to indirect prompt injection attacks, whereby malicious instructions are hidden within manipulated documents and interpreted by the LLM as commands instead of neutral data.
• A comprehensive evaluation was conducted across six attack objectives, including payment fraud and summary falsification, five delivery vectors (like free-text and metadata), five defensive strategies, and four LLMs, involving over 3,000 attack and 1,000 le...
• Defensive measures such as datamarking, structured queries, and guardrails effectively reduce attacks involving imperative instructions but are largely ineffective against summary falsification that substitutes plausible false data, revealing a critical pro...
• Attack success rates do not consistently correlate with potential administrative harm; for example, payment fraud and rule-exfiltration attacks present moderate success but high harm potential, necessitating risk assessments that balance both metrics.

Abstract

Long-running LLM agents are stateful: beyond the transcript they accrete compressed summaries, plaintext memory, pending tool plans, and, under every serving API, a KV cache. Yet today's "forget" operations delete a plaintext memory record and stop, leaving every artifact derived from the revoked information intact. We formalize execution-state unlearning: after a forget request, the agent must behave as if it had never observed the target. Modeling the runtime as a deterministic transition system, we prove that the pre-target trajectory prefix is shared with this counterfactual world for free, that the post-target suffix is irreducibly tainted without token-level attribution, and that exact unlearning requires at least $T-τ+1$ recomputed transitions, where $τ$ is the target's injection step. Provenance-Guided Selective Replay attains this bound as a cross-layer contract spanning prompt, compressed memory, and cache: a provenance graph locates the injection point, checkpoint restoration reduces to cropping the KV cache, and sanitized replay regenerates the counterfactual suffix. Audited with elicitation, stochastic, and string-free behavioral tests across three agent suites, nine baselines, and three model families, memory deletion leaves leakage unchanged, instruction-based forgetting collapses under elicitation (Leak@probes = 1.00), and source redaction still acts on a revoked preference in 80% of episodes, while selective replay is indistinguishable from a full reset at up to 9x fewer recomputed tokens.

Bullet Summary

• Long-running LLM agents maintain complex, multilayered state involving transcripts, compressed summaries, plaintext memory, pending tool plans, and KV caches, making forgetting operations challenging.
• Existing forget methods only delete explicit plaintext memory records but fail to erase derived artifacts such as summaries and cached states, resulting in persistent leakage of revoked information.
• The paper formalizes 'execution-state unlearning', modeling the agent's runtime as a deterministic transition system, and defines the goal as making the agent behave as if it never observed the target information.
• Theoretical analysis establishes that prefix states before the target injection point can be reused ('free prefix'), but exact unlearning requires replaying and recomputing all transitions after the injection step due to tainted suffix states.
• A novel method, Provenance-Guided Selective Replay, efficiently implements unlearning by using a provenance graph to identify the injection point, restoring checkpoints, cropping the KV cache, and replaying sanitized inputs ignoring the target data.

Abstract

The role of AI pair programmers has expanded from local code completion to active participation in the development environment. Contemporary tools can interpret repository context, edit multiple files, call package managers, execute terminal commands, and communicate with external services. This review synthesizes security evidence concerning GitHub Copilot, ChatGPT-based coding, code large language models, Cursor-style agentic editors, command-line coding agents, and Model Context Protocol ecosystems. A protocol-driven search, supplemented by backward and forward snowballing, covered literature and technical evidence available through 12 July 2026. The verified corpus comprised 216 verified records, including peer-reviewed studies, preprints, benchmarks, standards, and clearly identified technical disclosures. The evidence supports useful roles in vulnerability discovery and repair, test generation, and secure-coding guidance, but it also documents recurring injection flaws, unsafe memory and file handling, weak cryptography, authentication errors, secret exposure, hallucinated dependencies, and incomplete patches. Whether these weaknesses persist depends partly on human factors, including expertise, prompt framing, review effort, automation bias, and the authority users assign to the assistant. Most generated-code defects remain familiar CWE classes. Agentic systems add orchestration-level concerns: indirect prompt injection, context poisoning, tool and protocol supply-chain attacks, permission amplification, approval spoofing, cross-file persistence, and autonomous execution. We synthesize these findings in a unified taxonomy, an evidence-based threat model, a lifecycle security model, and a continuous-assurance architecture built on structured context, least privilege, sandboxing, provenance, conventional SAST/SCA/secret scanning, security tests, and mandatory human authorization for high-impact actions. Without such governance, faster production may be offset by accumulating validation debt and downstream incident risk.

Bullet Summary

• AI pair programmers have evolved into agentic environments capable of multi-file edits, terminal commands, and interacting with external services, enhancing development productivity but introducing new security challenges.
• The review analyzes 216 verified records on security vulnerabilities related to AI-generated code and agentic IDEs, identifying classic and emerging threats such as injection flaws, weak cryptography, prompt injection, and context poisoning.
• Human factors like expertise, prompt design, and review diligence critically influence security outcomes; automation bias and confidence in AI outputs can increase risk if not properly managed.
• AI-generated code commonly exhibits known vulnerability classes (e.g., CWE categories) but agentic orchestration adds complex risks like permission amplification, approval spoofing, and persistent malicious context.
• Large language models assist in vulnerability detection and repair, offering benefits over static analyzers, yet fully automated repair is still unreliable and requires human validation.

Abstract

Agentic social networks expose autonomous agents to large volumes of potentially malicious content, including prompt injection, social engineering, and unsafe execution requests. Large Language Model (LLM) judges can detect such content, but applying them at platform scale is expensive. We study the extent to which lightweight embedding-based prefilters can reduce this cost while retaining most of the judge’s unsafe predictions. We annotate 10,000 Moltbook posts and comments with a frontier LLM judge, assigning a binary safety verdict, severity level, malicious intent taxonomy labels, and Open Worldwide Application Security Project (OWASP) AI risk codes. Of the LLM-annotated samples, 9.1% were unsafe at severity 3 or above. Two human annotators with high inter-rater agreement (Cohen’s κ = 0.823) exhibited moderate agreement between their adjudicated labels and the LLM ratings (κ = 0.578). We compare centroid-based cosine prefilters over 3 off-the-shelf encoders against a supervised trained classifier. Reducing the input window size raises average precision by up to 6.3 percentage points. At an operating point calibrated to 0.80 recall, the projected cost of scanning 787,226 messages falls from $7,085 by 56.0% (MiniLM-L12-v2), 62.6% (BGE-M3), and 74.0% (trained classifier), while MiniLM runs roughly 32 times faster than the classifier. Prefiltering can therefore make frontier-judge screening affordable at scale, although jailbreak content remains the hardest category to retrieve.

Bullet Summary

• Agentic social networks expose autonomous Large Language Model (LLM) agents to harmful content such as prompt injections, social engineering, and unsafe requests, necessitating scalable safety screening methods.
• The study develops and evaluates embedding-based lightweight prefiltering approaches to efficiently reduce the volume of messages needing expensive LLM safety judgments while maintaining high recall of unsafe content.
• A dataset of 10,000 Moltbook social media posts and comments was annotated using a frontier LLM judge with binary safety verdicts, severity levels, malicious intent taxonomies, and OWASP AI risk codes; 9.1% of samples were unsafe at severity 3 or above.
• Two human annotators showed strong agreement with each other (κ = 0.823) and moderate agreement with the LLM judge (κ = 0.578), validating the annotation quality and complexity of the task.
• The study compares centroid-based cosine similarity prefilters using embeddings from three off-the-shelf encoders (MiniLM-L12-v2, BGE-M3, etc.) against a supervised trained classifier for unsafe content detection.

Abstract

Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection can achieve near-perfect attack success rates (ASRs). In the image domain, however, existing visual prompt injection methods are substantially less effective in attacking frontier commercial VLMs for materially harmful behavior. Achieving such outputs is hard because it requires a long and/or format-compliant target string, such as a precise, parseable native tool call with exact function names and arguments. We present Repeat-After-Me, a black-box adaptive visual prompt injection attack that can reveal personally identifiable information or make malicious tool calls. Across both open-weight and commercial frontier VLMs, including Qwen3.6-27B and GPT-5.5, our method achieves ASRs exceeding 80% and 47%, respectively, under a realistic setting in which the benign user prompt is semantically unrelated to the injected task and does not verbally authorize it. In our evaluation, injections optimized on one surrogate retain 43-46% of the original ASR on two commercial victims, and cross-sample transferability retains 64-66% of the original ASR on those two models. We test our attack in a real-world OpenClaw agent: in a default OpenClaw Discord deployment, an untrusted user can use a minimally injected image to overwrite TOOLS.md, enabling future sensitive behaviors like remote code execution and secret exfiltration. We show our new attack vector works in cases where adaptive textual prompt injection fails. We discuss potential defenses.

Bullet Summary

• Prompt injection poses a critical security threat to AI agents processing untrusted external data, especially through visual channels in vision-language models (VLMs).
• Existing black-box visual prompt injection (VPI) attacks have limited success in inducing harmful, precise behaviors like private information disclosure or malicious tool calls in frontier commercial VLMs.
• The paper introduces Repeat-After-Me (RAM), a novel black-box adaptive visual prompt injection attack that overlays attacker-chosen textual prompts on images, explicitly instructing VLMs to start their responses with target malicious outputs.
• RAM achieves high attack success rates exceeding 80% on open-weight VLMs and 47% on commercial frontier VLMs (e.g., Qwen3.6-27B, GPT-5.5) under realistic benign user prompts unrelated to the injected task.
• An adaptive optimization strategy and a reusable Attack Library of successful injections enable RAM to maintain significant transferability across different VLMs and user prompts without direct querying or knowledge of benign prompts.

Abstract

Large language models (LLMs) are becoming integral to web applications and browser agents, transforming online interactions while introducing new attack vectors and reshaping longstanding web vulnerabilities. Classical threats such as cross-site scripting (XSS) can be amplified through LLM-mediated interactions, while LLM-specific vulnerabilities can propagate across web applications, introducing attacks such as prompt injection. Securing modern web systems therefore requires understanding interactions between traditional and LLM-specific threats across the system lifecycle. Unlike prior surveys treating web and LLM security separately, this survey provides a unified analysis of how LLMs amplify web vulnerabilities across client-side, server-side, and pipeline layers while evaluating defenses and their limitations. The analysis examines extending NIST and ISO/IEC AI security frameworks to the security needs of LLM-enabled web environments. Three unresolved challenges are identified: adversarial natural-language instructions, autonomous agent security, and post-deployment security through continuous monitoring and adaptation. An LLM-aware monitoring and control framework is proposed, integrating semantic input validation, prompt integrity protection, output isolation, agent governance, and runtime monitoring. This unified perspective characterizes the evolving threat landscape and outlines future directions for secure AI-enabled web systems.

Bullet Summary

• Large language models (LLLs) integrated into web applications introduce novel attack vectors that amplify traditional web vulnerabilities such as cross-site scripting (XSS) through LLM-mediated interactions.
• LLM-specific vulnerabilities, including prompt injection and embedding inversion attacks, affect client-side, server-side, and backend pipeline layers, linking classical web security threats with AI-specific risks.
• Current security frameworks (e.g., OWASP, NIST, ISO) inadequately address the compounded risks of LLM integration, necessitating an extension and unification of these frameworks for LLM-enabled web environments.
• The authors propose a taxonomy aligning OWASP LLM Top 10 risks with traditional web vulnerabilities (MITRE CWE categories), enabling a structured analysis of attack surfaces introduced by LLMs.
• Three unresolved security challenges identified are adversarial natural-language instructions, autonomous agent security with tool access, and post-deployment continuous monitoring and adaptation.

Abstract

Abstract unavailable from OpenAlex metadata.

Bullet Summary

• LLM-integrated applications combine language models with retrieval, memory, tools, and delegated credentials, introducing complex security challenges as untrusted inputs can cross trust boundaries and gain operational authority.
• Existing research predominantly focuses on prompt injection attacks and taxonomy of risks but lacks practical frameworks to translate these threats into verifiable and testable security controls in deployed systems.
• The paper proposes a trust-boundary security assurance framework that models LLM-agentic systems as mediated-authority architectures with defined threat surfaces, trust boundaries, and attack paths to systematically analyze security risks.
• A six-stage assurance process is developed: system decomposition, authority mapping, adversarial scenario construction, control verification, containment and detection analysis, and evidence-based reporting to guide comprehensive security assurance.
• Security controls are recommended to bind actions strictly to authenticated principals, resources, and explicit policies, moving away from reliance on model confidence or prompts for authorization decisions.

Abstract

Reusable agent skills extend large language model (LLM) agents with task procedures, tool-use guidance, and output constraints. Yet these skills also act as externalized behavioral policies, which create a supply-chain risk: a third-party skill may preserve the declared task and valid output interface while covertly redirecting agent decisions toward an undisclosed objective. We formalize Skill Policy Integrity, which requires a Skill-induced policy to remain aligned with its declared functionality and the user-authorized objective. We further present SkillShift, a constrained black-box framework for covert policy steering without explicit target command injection or task hijacking. It combines semantically plausible policy edits with hierarchical validation, failure-guided optimization, and strategy compression to preserve effectiveness, output validity, transferability, and inconspicuousness. We instantiate this threat in agentic commerce and software dependency use, with SkillShift achieving attacker-favored selection rates of 81.33% and 63.33% while maintaining a 100% utility-preserving rate. The frozen policies also transfer without further optimization across heterogeneous LLM backends and agent environments. Moreover, the evaluated scanners fail to detect the constructed skills, motivating behavioral auditing of reusable skills as agent policy artifacts.

Bullet Summary

• Reusable agent skills for large language model (LLM) agents enhance capabilities but introduce supply-chain risks by potentially covertly steering agent policies toward attacker objectives without task alteration.
• The paper formalizes Skill Policy Integrity, requiring that skill-induced policies remain aligned with declared functionalities and authorized objectives to prevent covert manipulation.
• SkillShift is proposed as a black-box, constrained framework that covertly steers multi-agent policies by semantically editing skills to subtly influence candidate selection while preserving output validity and task fidelity.
• SkillShift constructs a structured, context-consistent strategy appended to base skills using global principles, task-specific rules, tie-breakers, and semantic examples to achieve covert policy steering without explicit instruction injection.
• Experimental evaluations in agentic commerce and software dependency selection tasks show SkillShift achieves high attacker-favored selection rates (81.33% and 63.33%) while preserving 100% utility and output validity, with strategies transferable across he...

Abstract

Persistent AI agents construct autobiographical state through reflection, retrieval, and consolidation. Persistence changes availability, not epistemic standing: stored or retrieved material is not thereby supported. Untrusted inputs, prompt injections, and model inferences can therefore enter persistent state and later be presented as agent history or user commitments. We specify typed provenance and assertion guardrails for autobiographical assertion boundedness, a system-relative release property requiring governed statements about the agent, user, or named relationships to satisfy accepted-evidence, temporal-validity, and disclosure policies. A typed provenance graph separates origin, dependency lineage, epistemic role, validity, and disclosure scope. A resolver evaluates authorized state projections and returns one evidential status, orthogonal conflict, staleness, and withholding flags, and a protected decision witness. A generate-verify-revise mediator then checks candidate semantic units before release and renders policy-authorized status responses. Under explicit assumptions about extraction, predicate correctness, resolution soundness, view declassification, and channel mediation, we prove a conditional assertion-boundedness contract. In an executable suite of 24 hand-authored conformance cases, typed mediation passed none of 19 unsafe opportunities unqualified while preserving all five supported controls. The flat/prior and source-tag comparison rules released 19/19 and 18/19 unsafe candidates, respectively. These results validate the encoded resolver and mediator obligations; they do not constitute an end-to-end evaluation of language models or retrieval systems.

Bullet Summary

• Persistent AI agents build autobiographical states through reflection and retrieval, but storing information does not guarantee its epistemic support or truthfulness, posing risks of untrusted inputs and prompt injections influencing stored claims.
• The paper introduces typed provenance and assertion guardrails to enforce autobiographical assertion boundedness, ensuring that agent assertions satisfy accepted-evidence, temporal-validity, and disclosure policies before being released.
• A typed provenance graph structures claims and evidence by origin, dependency lineage, epistemic role, validity intervals, and disclosure scope, enabling a resolver and a generate-verify-revise mediator to evaluate and authorize assertions safely.
• The approach enforces three distinct surfaces—state admission, authorized projection, and assertion mediation—to separate storage from support and prevent unsupported claims from being asserted externally.
• The authors formalize a typed autobiographical claim model, design a procedural resolver and assertion mediation pipeline with conditional guarantees, and validate the system with an executable conformance suite of 24 threat cases.

Abstract

Skill selection is a key stage in LLM-agent workflows, determining which installed skill should handle a user request. Existing attacks on this stage primarily rely on explicit prompt injection or instruction-level steering, which can expose recognizable manipulation signals. In this work, we identify a new implicit attack surface for skill selection: even when the user prompt and skill description appear benign in isolation, their semantic relationship can still be strategically shaped to favor an attacker-chosen skill. Based on this observation, we present Implicit Skill-Selection Manipulation via Semantic Matching (ISM), which jointly shapes target-skill metadata and reusable prompts to manipulate skill selection without explicit selection instructions. Specifically, we develop a three-stage strategy to broaden semantic coverage, strengthen target distinctiveness, and preserve natural prompt wording. Across four task domains and eight selector models, ISM increases the average target-selection rate (TSR) from 15.2% to 63.5%. In a matched comparison, ISM achieves a 73.5% TSR, only 9.8 percentage points below Explicit Steering. Human reviewers block ISM in only 2.9% of judgments, versus 91.4% for Explicit Steering, while five LLM-based inspectors pass ISM at an average rate of 82.9%, versus 37.4% for Explicit Steering. Moreover, ISM remains effective against PPL-W, Llama Prompt Guard 2, and PIGuard.

Bullet Summary

• The paper addresses the vulnerability of skill selection in large language model (LLM) agent workflows, where semantic matching determines which installed skill handles a user request.
• It introduces Implicit Skill-Selection Manipulation via Semantic Matching (ISM), a novel attack method that covertly influences skill selection by manipulating semantic relationships between benign-looking user prompts and skill metadata, without explicit i...
• ISM employs a three-stage technique: task-related cue construction to generalize semantic coverage, customized technical-hook injection to distinguish the target skill, and semantic-constrained prompt naturalization to maintain natural wording and evade det...
• Experiments across eight LLM models, four task domains, and 56 installed skills show ISM increases target skill selection rates from an average of 15.2% to 63.5%, nearly matching explicit steering robustness, with low detection by humans and automated defen...
• ILL exploits the semantic compatibility between prompts and skill descriptions, measured via MiniLM embeddings, improving robustness across various prompt formulations and resisting state-of-the-art defenses like PPL-W and Llama Prompt Guard 2.

Abstract

The rapid migration of Large Language Models (LLMs) from conversational interfaces to autonomous multi-agent software engineering systems has exposed profound security vulnerabilities. When autonomous agents operate across heterogeneous topologies—spanning cloud-hosted reasoning engines and local execution environments—they are acutely vulnerable to indirect prompt injection, tool-call hijacking, privileged command escalation, and memory poisoning. Conventional boundary defenses (such as input sanitizers and prompt wrappers) fail to address lateral privilege escalation between collaborating agents. To resolve this critical architectural vulnerability, this paper introduces AgentShield, a zero-trust runtime verification and guardrail framework designed for decentralized multi-agent systems operating over the Model Context Protocol (MCP). AgentShield enforces continuous, non-bypassable policy verification across all intra-agent communications and system tool dispatches. The framework incorporates: (1) an inline bidirectional semantic interceptor that evaluates agent intents before system execution, (2) a multi-lingual token triage engine capable of detecting adversarial jailbreaks in low-resource and code-switched dialects, (3) a cryptographically signed persistent shared ledger ensuring tamper-evident state continuity, and (4) an automated capability attenuator for operating system and file operations. We evaluate AgentShield across 1,500 adversarial scenarios covering multi-step tool-injection benchmarks and real-world developer workflows. Empirical results demonstrate that AgentShield mitigates 98.4% of prompt injection and tool-escalation attacks while introducing less than 11.8ms of median runtime latency overhead. The core architecture is validated via two production-grade open-source packages released on the Python Package Index (PyPI): prema-agentshield and gemini-antigravity-bridge.

Bullet Summary

• Rapid adoption of Large Language Models (LLMs) as autonomous multi-agent AI systems has introduced serious security vulnerabilities, particularly in heterogeneous environments combining cloud and local execution.
• Existing boundary defense mechanisms (e.g., input sanitizers, prompt wrappers) do not effectively prevent lateral privilege escalation among collaborating agents.
• AgentShield is proposed as a zero-trust runtime verification and guardrail framework specifically designed for decentralized multi-agent systems using the Model Context Protocol (MCP).
• AgentShield features a bidirectional semantic interceptor that inspects agent intents prior to system execution to prevent malicious actions.
• It includes a multi-lingual token triage engine that detects adversarial jailbreaks even in low-resource and code-switched dialects, enhancing robustness against prompt injection attacks.

Abstract

The widespread adoption of systems based on Large Language Models has made the reliable detection of prompt injection attacks a critical requirement. However, high performance achieved on training and test splits generated from the same data source does not guarantee that models can generalize to prompts from different sources. In this study, a leak-aware cross-dataset evaluation framework is presented to examine the robustness of classical machine learning and Transformer-based prompt injection detection models in the face of data source changes. During the data preparation process, empty records, duplicate prompts, conflicting labels, and text overlaps between datasets were checked. In this context, 38,184 duplicate records and 14 instances with conflicting labels were removed, and the 198 common prompts identified between the training and external test sets were removed only from the training set. Using WordHash and CharHash representations, SGD Logistic and Linear SVM models, as well as DistilBERT and DeBERTa-v3-small, were evaluated on an internal dataset consisting of 426,073 cleaned requests; the models were also tested on an independent dataset of 5,000 examples. While the models achieved performance in the range of approximately 0.997–1.000 in the internal evaluation, significant performance losses were observed in the external evaluation. DeBERTa-v3-small delivered the most balanced results, with an accuracy of 0.7360, a balanced accuracy of 0.7320, a Macro-F1 of 0.7284, and an attack sensitivity of 0.7119. The domain classifier achieved an ROC-AUC of 0.9770 hence indicating a remarkable shift in the distribution across data sources. The results show that internal validation results are not enough for prompt injection detection. Independent external validation, data leakage verification and domain shift analysis should be fundamental parts of reliable model evaluation.

Bullet Summary

• The study addresses the critical problem of detecting prompt injection attacks on Large Language Model (LLM)-based systems, highlighting the challenge of model generalization across data sources.
• A novel leakage-aware cross-dataset evaluation framework is presented, involving rigorous dataset cleaning to remove duplicates, conflicting labels, and overlapping prompts to prevent data leakage.
• Classical machine learning models (using WordHash and CharHash feature representations with logistic regression and linear SVM) and Transformer-based models (DistilBERT and DeBERTa-v3-small) were trained and validated on a large internal dataset (426k promp...
• Internal validation results indicated near-perfect detection performance (accuracy and macro-F1 ~0.999), but significant performance drops were observed during external validation, with the best model (DeBERTa-v3-small) achieving balanced accuracy around 0....
• Domain classification experiments indicated strong distribution shifts between internal and external datasets (ROC-AUC ~0.98), underscoring dataset shift as a core challenge to prompt injection detection generalizability.

Abstract

Abstract unavailable from OpenAlex metadata.

Bullet Summary

• The paper addresses security challenges in large language model (LLM) autonomous agents, focusing on mitigating indirect prompt injection (IPI) attacks arising from ambient authority and instruction-data conflation.
• It introduces Mastyf Guard, a deterministic-first security architecture utilizing an external reference monitor to enforce capability-based controls with least privilege and typed relational argument invariants, ensuring zero probability of unauthorized too...
• Mastyf Guard integrates a multi-tier defense pipeline combining fast syntactic checks, neural semantic audits, and capability verification, achieving 99.33% threat recall and over 91% precision with minimal latency and low hardware requirements.
• The Cognitive Harvard Architecture conceptual framework is proposed to separate instructions and data at execution boundaries, addressing the Cognitive Von Neumann Conflation vulnerability in AI systems by dividing processing into Cognitive Ingestion and Pr...
• The system implements dynamic-taint information-flow control to prevent unauthorized cross-tool data exfiltration, successfully blocking all tested adversarial attempts with negligible latency impact.

Abstract

Persistent AI agents construct autobiographical state through reflection, retrieval, and consolidation. Persistence changes availability, not epistemic standing: stored or retrieved material is not thereby supported. Untrusted inputs, prompt injections, and model inferences can therefore enter persistent state and later be presented as agent history or user commitments. We specify typed provenance and assertion guardrails for autobiographical assertion boundedness, a system-relative release property requiring governed statements about the agent, user, or named relationships to satisfy accepted-evidence, temporal-validity, and disclosure policies. A typed provenance graph separates origin, dependency lineage, epistemic role, validity, and disclosure scope. A resolver evaluates authorized state projections and returns one evidential status, orthogonal conflict, staleness, and withholding flags, and a protected decision witness. A generate-verify-revise mediator then checks candidate semantic units before release and renders policy-authorized status responses. Under explicit assumptions about extraction, predicate correctness, resolution soundness, view declassification, and channel mediation, we prove a conditional assertion-boundedness contract. In an executable suite of 24 hand-authored conformance cases, typed mediation passed none of 19 unsafe opportunities unqualified while preserving all five supported controls. The flat/prior and source-tag comparison rules released 19/19 and 18/19 unsafe candidates, respectively. These results validate the encoded resolver and mediator obligations; they do not constitute an end-to-end evaluation of language models or retrieval systems.

Bullet Summary

• Persistent AI agents build autobiographical state via reflection, retrieval, and consolidation, but persistence affects availability without guaranteeing epistemic support for stored or retrieved information.
• Untrusted inputs, prompt injections, and model inferences can infiltrate the persistent state and be misrepresented as valid agent history or user commitments, posing security challenges.
• The paper introduces typed provenance and assertion guardrails for autobiographical assertion boundedness, enforcing system-relative release properties that mandate governed statements meet evidence acceptance, temporal validity, and disclosure policies.
• A typed provenance graph distinctly categorizes origin, dependency lineage, epistemic role, validity, and disclosure scope of assertions.
• A resolver mechanism evaluates authorized state projections producing evidential status, conflict, staleness, withholding indicators, and a protected decision witness to support secure assertions.

Abstract

Real-world, high-profile AI agent harnesses often rely on vendor-proprietary or opaque designs for context assembly, leaving the sources and underlying logic of assembled context poorly understood and the resulting security risks largely unexplored. In this paper, we present the first systematic analysis of context assembly designs in real-world AI agent harnesses. We study and uncover how an agent harness is designed to collect and assemble context from diverse sources, and identify a set of practical attack vectors arising from these designs. Our analysis brings to light two novel categories of attacks in the context assembly of real-world harnesses: (1) MessageRole Context Privilege Escalation (M-CPE), which occurs when attacker-controlled content originating from a low-privileged context is incorporated into a higher-privileged message role. (2) Cross-Scope Context Privilege Escalation (X-CPE), which occurs when attacker-controlled content persists beyond the context in which it was introduced. We performed a systemic security analysis of the CPE attacks against 12 real-world agent harnesses, including Claude Code and Codex. The resulting consequences include full agent compromise, remote code execution, denial of service, and manipulated tool or skill invocations, etc.

Bullet Summary

• This paper analyzes context assembly designs in real-world AI agent harnesses, revealing how context is aggregated from multiple heterogeneous sources that are often vendor-proprietary and opaque, posing security risks.
• It identifies two novel categories of Context Privilege Escalation (CPE) attacks: Message-Role CPE (M-CPE), where low-privileged content is escalated into higher-privileged message roles, and Cross-Scope CPE (X-CPE), where malicious content persists beyond...
• A systematic security evaluation was conducted on 12 popular AI agent harnesses including Claude Code and Codex, uncovering 282 vulnerable context sources exploitable via 16 new attack vectors leading to severe consequences such as full agent compromise, re...
• The researchers developed CORA, an LLM-assisted automated tool that performs multi-stage analysis—including static and runtime instrumentation—to identify vulnerable context sources, privilege roles/scopes, and to validate exploit paths through reproducible...
• The work highlights the complex privilege assignment and hierarchical context structure in AI agents, including multiple message roles (system, user, assistant, tool) and varying scopes (user, project, session), which attackers exploit to escalate privileges.

Abstract

Real-world, high-profile AI agent harnesses often rely on vendor-proprietary or opaque designs for context assembly, leaving the sources and underlying logic of assembled context poorly understood and the resulting security risks largely unexplored. In this paper, we present the first systematic analysis of context assembly designs in real-world AI agent harnesses. We study and uncover how an agent harness is designed to collect and assemble context from diverse sources, and identify a set of practical attack vectors arising from these designs. Our analysis brings to light two novel categories of attacks in the context assembly of real-world harnesses: (1) MessageRole Context Privilege Escalation (M-CPE), which occurs when attacker-controlled content originating from a low-privileged context is incorporated into a higher-privileged message role. (2) Cross-Scope Context Privilege Escalation (X-CPE), which occurs when attacker-controlled content persists beyond the context in which it was introduced. We performed a systemic security analysis of the CPE attacks against 12 real-world agent harnesses, including Claude Code and Codex. The resulting consequences include full agent compromise, remote code execution, denial of service, and manipulated tool or skill invocations, etc.

Bullet Summary

• Real-world AI agent harnesses aggregate contextual data from diverse, often opaque sources with varying trust levels, creating complex privilege role hierarchies that assign different privileges to context segments.
• The paper identifies two novel attack categories exploiting these assemblage designs: Message-Role Context Privilege Escalation (M-CPE), where attacker-controlled low-privileged content infiltrates higher privilege message roles; and Cross-Scope Context Pri...
• A comprehensive taxonomy of 16 new context privilege escalation attack vectors is constructed, categorized into diverse context sources exploitation, context markup manipulation, and abuse of context assembly logic.
• The authors developed CORA, an automated analysis and exploit tool that identifies, validates, and exploits context privilege escalation (CPE) vulnerabilities by analyzing 12 popular agent harnesses—including Codex and Claude Code—revealing widespread vulne...
• Experimental evaluation confirmed 282 context sources across roles and scopes, with CORA detecting 1761 candidate CPE attack paths; behavioral validation yielded high exploitability rates across multiple LLM models, demonstrating real-world impact potential.

Abstract

Long-term memory can turn untrusted external content into persistent influence over an LLM agent's future decisions, creating the threat of indirect memory poisoning. A successful attack must survive a multi-stage pipeline comprising memory writing, retrieval, and utilization. Existing attacks largely rely on intra-stage optimization, optimizing individual stages in isolation while overlooking inter-stage coupling. Specifically, these stages impose different requirements on the same poisoning content, and each stage operates on the transformed output of its predecessor. Consequently, optimizing one stage may undermine the effectiveness of other stages, while upstream transformations may erase improvements intended for downstream stages. Indirect memory poisoning should therefore be viewed as an end-to-end optimization problem. Based on this insight, we present \textsc{PipePoison}, which collects fine-grained stage feedback from local shadow systems, uses chain-structured losses to identify and optimize the stage bottlenecking end-to-end success, and applies stability-calibrated stage and configuration weights to improve transferability. Across three agent frameworks and four memory mechanisms, \textsc{PipePoison} improves attack utilization rate by 19.1 percentage points. Even on fully unseen victim configurations, it outperforms the strongest baseline by 16 percentage points and remains effective under eight representative defenses.

Bullet Summary

• Memory Agent MINJA ER-MIA Zombie Sleeper MemMorph GhostWriter PIPEPOISON WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR WSR RSR@5 AUR Langmem LangGraph 62% 88% 50% 47% 74% 36% 33% 78% 26% 43% 74% 37% 69% 89% 61% 48% 72%...
• 4 Evaluation 4.1 Experimental Setup Agent and Memory Systems.We evaluate PIPEPOI- SONon three real-world open-source agent frameworks (GitHub stars in parentheses)—LangGraph (39.8K) [17], CrewAI (57.5K) [4], and OpenAI Agents (28.9K) [28]— combined with fou...
• Transferable End-to-End Optimization for Indirect Long-Term Memory Poisoning in LLM Agents Chuanchao Zang1 Jianing Wang1 Wenyu Chen1 Xiangtao Meng1 Li Wang1 Xinyu Gao1 Zheng Li1,2,3* Shanqing Guo1,2,3* 1School of Cyber Science and Technology, Shandong Unive...
• Based on whether the attacker expects the poisoning content to affect agents with a particular configuration, we consider two scenar- ios.(1) In the matched scenario , the attack is designed for a particular expected configuration and takes effect if an age...
• Agent Prompt You are a helpful agent equipped with external tools and long-term memory.

Abstract

Abstract unavailable from Crossref metadata.

Bullet Summary

• Introduces a novel approach for threat detection and attribution in multi-agent large language model (LLM) systems by analyzing system logs instead of relying solely on prompts.
• Addresses the challenge of identifying malicious or compromised agents within complex multi-agent LLM environments where traditional prompt-based methods fall short.
• Develops a log-based detection framework that monitors inter-agent communications and system behaviors to detect anomalous patterns indicative of security threats.
• Implements evaluation experiments simulating various attack scenarios, demonstrating the method's effectiveness in accurate threat identification and attribution.
• Highlights the importance of detailed logging and behavioral analysis to enhance the security posture of cooperative multi-agent LLM systems.

Abstract

Agentic AI systems are increasingly deployed to process sensitive data at inference time, such as healthcare records or financial documents assembled into a hidden \emph{context} before the system answers. Prior work has studied privacy risks primarily through \emph{jailbreaking} attacks that induce models to directly disclose sensitive content, but has largely overlooked the agentic setting where the context is assembled by the agent's own tool calls. We show that the agents we evaluate remain vulnerable to hidden-context leakage despite the controls we test against them, namely an instruction not to disclose the context, logit suppression, and context dilution. For instance, a web-browsing agent answering benign user queries still carries exploitable signals about records silently loaded into its context. We introduce and formalize \emph{context-inference attacks} through a security game and evaluate three settings under decreasing attacker knowledge and increasingly indirect delivery of the context: a known context, an unknown context, and a context the agent retrieves through its own tool calls. We distinguish a grey-box setting, in which the target model is used to score observations, from black-box settings in which the attacker scores with a surrogate it controls. We further characterize how leakage varies with query budget, context size, and target-model size. A single attack carries through all three settings without modification, reaching $100\%$ ASR on small candidate sets and $63\%$ at $1024$ candidates against a known context, $78.9$ AUROC when the template and surrounding records are unknown, $92.5$ AUROC when a 14B surrogate scores a 32B target, and $81.8$ AUROC when the records arrive as an agent's retrieval returns, against chance rates of $1/|\mathcal{Z}|$ and $50$ respectively.

Bullet Summary

• Agentic AI systems process sensitive data by assembling hidden contexts at inference time, which introduces unique privacy vulnerabilities beyond traditional jailbreak-style direct content disclosures.
• The paper formalizes 'context-inference attacks,' which exploit subtle statistical signals in benign model outputs to infer hidden contexts without explicit disclosure or adversarial queries.
• A unified and replay-based likelihood inference framework is introduced, allowing attackers to test hypotheses about the hidden context using benign, target-agnostic queries across varying threat models of attacker knowledge and context delivery.
• Experiments demonstrate high effectiveness of context-inference attacks across multiple large language and vision-language models, with attack success rates exceeding 80% AUROC even under black-box settings and without adversarial instructions.
• The attack framework is robust across three threat models: TM1 with known candidate sets and templates; TM2 with unknown templates and candidates; and TM3 where hidden context is agent-retrieved records rather than fixed text.

Abstract

Recently, large language model (LLM) agents, such as Codex, Claude Code, and OpenClaw, have become capable of planning and executing long-horizon tasks through repeated tool calls. This capability also creates new opportunities for prompt injection. Existing attacks either place the malicious objective in one explicit instruction, making it easy to detect, or distribute the intent across multiple execution stages, making successful completion unreliable. In this work, we propose ECLIPSE, a self-evolving and stealthy prompt-injection framework for long-horizon agentic systems. ECLIPSE combines direct user-prompt injection with indirect tool-side injection through two components. On the one hand, Stealthy Attack Trajectory Synthesis uses a sandbox to generate and iteratively verify candidate tool chains, then renders a verified chain as a natural one-shot prompt to serve as the direct instruction. Then, Tool-Chain Steering transfers this plan to the target environment through Static Workflow Encoding (SWE), which embeds state-transition cues in target-tool descriptions, and Dynamic Trajectory Correction (DTC), which supplies corrective signals when execution deviates from the planned chain. To enable systematic evaluation, we further introduce LASE-Bench, a long-horizon agent-safety benchmark with 120 malicious tasks and 198 unique tools; 96.7% of its tasks make at least five tool calls. The experimental results show that ECLIPSE is highly effective: it achieves up to 96.7% attack success without defense and 69.2% under the common safety filter, exceeding the strongest baseline by 27.5% in the defended setting. Evaluations against representative defenses further show that existing safeguards do not reliably defend it, which raises the need for more effective defenses.

Bullet Summary

• ECLIPSE is a novel prompt injection attack framework targeting long-horizon LLM agents, combining direct user-prompt injection and indirect tool-side injection for stealthy and reliable multi-step attacks.
• The framework addresses stealth and fidelity challenges by synthesizing stealthy attack trajectories via a sandbox environment and steering tool execution through Static Workflow Encoding (SWE) and Dynamic Trajectory Correction (DTC).
• ECLIPSE generates and verifies plausible multi-tool attack chains that appear natural and inconspicuous when encoded as one-shot prompts, enhancing stealth against detection mechanisms.
• Static Workflow Encoding embeds a chained workflow into tool descriptions to guide execution plausibly, while Dynamic Trajectory Correction monitors and corrects deviations at runtime, improving attack robustness.
• The authors introduce LASE-Bench, a comprehensive long-horizon security benchmark with 120 malicious tasks and 198 unique tools, to systematically evaluate prompt injection attacks on LLM agents.

Abstract

LLM-based agent systems increasingly adopt skill-based architectures to reduce repetitive reasoning costs and improve stable, efficient task execution. Recent studies propose self-evolving agents that autonomously generate, refine, and reuse skills from past experiences to enable continuous capability evolution. However, autonomous skill evolution introduces a new attack surface in which malicious capabilities are generated, stored, and reused as legitimate skills. In this paper, we define EvoSkill Injection as a threat model targeting the autonomous skill generation and evolution pipeline of self-evolving agents. We further propose SARGE (Red-teaming Autonomous Skill Generation and Evolution in self-evolving agents), a red-teaming framework for evaluating this threat model through iterative generation, escalation, and reinforcement interactions. To support our framework, we construct EvoSkillBench, a benchmark dataset of malicious interaction trajectories for inducing malicious skill formation in self-evolving agents, and introduce EvoSkillSafetyBench, a post-attack benchmark for evaluating whether injected malicious skills are subsequently retrieved and activated as harmful behaviors. Our evaluation shows that SARGE induces malicious skill formation and that injected skills are persistently stored and repeatedly activated, highlighting the risk of persistent capability corruption.

Bullet Summary

• The paper introduces EvoSkill Injection, a novel threat model where self-evolving agents autonomously generate, store, and reuse malicious skills due to adversarial interactions within their skill evolution pipelines.
• SARGE, a modular multi-agent red-teaming framework, is designed to evaluate EvoSkill Injection by iteratively generating, escalating, and reinforcing harmful skills through orchestrated attacker, judge, and target agent interactions.
• Two benchmark datasets are created: EvoSkillBench, containing adversarial interaction trajectories inducing malicious skill formation, and EvoSkillSafetyBench, assessing whether injected malicious skills are retrieved and activated, causing harmful behaviors.
• Experiments on multiple self-evolving agents (AutoSkill, Voyager, ExpeL) and large language models demonstrate high attack success rates, with malicious skills persistently stored and repeatedly activated, leading to sustained capability corruption and incr...
• The study reveals that malicious skills can transfer across diverse LLM architectures, emphasizing the cross-model persistence of these injected threats.

Abstract

Multi-agent applications delegate work across independently operated deployers. After an incident, a verifier must answer two questions: which deployer released the reported bytes, and whether each cross-deployer edge was authorized. Credentials establish who may act, but need not bind them to later output bytes or prove both deployers authorized a dynamically created edge. We present a two-layer attestation design for dynamic delegation without a shared authority, public log, or precommitted workflow. A trusted deployer runtime signs a hash of each released output; this records released bytes but does not prevent prompt injection. Ancestry evidence records edge authorization. Under a unified threat model, we compare a signed linked list, a Merkle-chain variant, and a co-signed DAG. The primitives are standard; the contribution is deployer-side binding and the evidence needed for the two questions. After child-key compromise, the single-signer designs permit an unauthorized parent binding, whereas the co-signed DAG rejects it because the parent must authorize the edge. Fixed adversary matrices and regression tests validate the composed verifier. On an Apple M1 Pro, ancestry-only checks take 24.3-499.2us per hop. In a live local multi-service workflow, a parent discovers the child's A2A Agent Card; the child calls an MCP tool and releases local-LLM output: all 30 signed-DAG tasks passed complete verification, while a controlled child-key-only claim was rejected; its mean end-to-end latency was 813.1ms versus 770.8ms without evidence. In a complementary three-availability-zone AWS deployment, all 1,000 valid co-signed-DAG paths verified; issuance averaged 3.651ms and complete verification 5.015ms. The cloud result excludes TLS/mTLS, KMS, and model-serving latency.

Bullet Summary

• Addresses the challenge of attesting which deployer released specific outputs and authorizing cross-deployer delegation edges in multi-agent systems without shared authority or public logs.
• Proposes a two-layer attestation design: Layer 1 binds outputs to deployers via signed hashes; Layer 2 provides ancestry evidence for delegation edge authorization.
• Compares three ancestry-binding methods: signed linked lists, Merkle-chain variants, and co-signed Directed Acyclic Graphs (DAGs), with the co-signed DAG uniquely requiring both parent and child signatures to resist unauthorized edge creation under child-ke...
• Employs a unified threat model considering message tampering, replay, and structural attacks, but excludes cryptographic breakage or trusted runtime compromise; includes formal defenses against adversarial manipulations of delegation paths.
• Implements cryptographic primitives Ed25519 and SHA-256 with deterministic JSON serialization to ensure consistent and tamper-evident records.

Abstract

As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user's perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent's behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79-12.01 percentage points over the strongest baseline.

Bullet Summary

• Indirect Prompt Injection (IPI) attacks pose a serious security threat to LLM agents by embedding malicious instructions into tool observations, causing unintended agent actions.
• The paper distinguishes between 'covert' and 'overt' IPI successes: covert attacks execute malicious tasks without alerting the user, while overt attacks reveal the injection in the agent's final response.
• A novel metric called Covert Success Rate (CSR) is introduced to complement the traditional Attack Success Rate (ASR), measuring how often attacks remain undetected by users, alongside Overt Success Rate (OSR).
• Analysis reveals that covert attacks are linked to the ReAct framework's behavior, where agents return control to the user task before responding, concealing the injected action.
• The authors propose ICoA (Induced Covert Attack), an IPI attack method that structures prompts to induce covert outcomes by executing injections and then steering the agent back to the user task.

Abstract

As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user's perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent's behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79-12.01 percentage points over the strongest baseline.

Bullet Summary

• Introduces the problem of indirect prompt injection (IPI) attacks on tool-using large language model (LLM) agents, where malicious instructions are embedded in external content to hijack agent behavior without user awareness.
• Distinguishes between overt and covert injection successes: overt where the injected action is revealed in the agent’s final response, and covert where the attack executes silently with no user-visible trace.
• Proposes a new metric, Covert Success Rate (CSR), to specifically measure the fraction of injection attacks that remain undetected by users, complementing the traditional Attack Success Rate (ASR).
• Analyzes agent behavior trajectories and finds that covert successes occur when the agent reverts back to the original user task after executing the injection, due to the ReAct prompt format's structure.
• Develops ICoA (Induced Covert Attack), an IPI attack method that uses a payload combining user framing, an injection goal, and a RETURN anchor to steer the agent back to the user task, thereby inducing covert injection success.

Abstract

LLM agents commonly use knowledge-based tools and access their underlying files, databases, and search indexes through tool invocation. This integration improves agents' ability to provide domain-specific services but also introduces the risk of tool-mediated knowledge extraction: source content exposed to an agent for legitimate responses may be progressively recovered from its outputs, enabling reconstruction of the knowledge source behind a target tool. This paper systematically investigates this risk and identifies two challenges introduced by tool invocation: tool-selection uncertainty, where an agent may invoke a competing tool instead of the target tool, and tool-argument compression, where fine-grained query information may be lost when the agent generates tool arguments. To tackle these challenges, we propose ToolSiphon, a query-only extraction attack that introduces two complementary signals: a target-discriminative signal, implemented through Tool Contrastive Analysis, to steer queries toward the target tool; and a response-grounded factual signal, implemented through Evidence Chained Feedback, to mitigate argument compression and progressively expand extraction coverage. Across three types of knowledge-based tools and six domain-specific datasets, ToolSiphon recovers 74.3% of source records on average when coarse-grained information about non-target tools is available, with 83.2% textual recovery and 90.2% semantic similarity. Even without such information, it recovers 66.3% of source records. ToolSiphon also remains effective against representative defenses and on three real-world agent platforms.

Bullet Summary

• This paper systematically investigates this risk and iden- tifies two challenges introduced by tool invocation: tool- selection uncertainty, where an agent may invoke a compet- ing tool instead of the target tool, and tool-argument com- pression, where fine...
• Target and Selected Tool Other and Not Selected Tool Step 2: Tool-Argument Generation Tool Name: symptom_checker Tool Functional Description: A tool designed to query disease-specific symptoms and mitigation measures.
• We append the following context to its system prompt: Additional System Prompt for Jail-IKEA The coarse-grained functionality of the target extrac- tion tool is: [TARGET TOOL FUNCTIONALITY] The coarse-grained functionalities of its relevant tools are: [RELE...
• Given the query “Find prior cases involving Acme’s tax dispute, ” the agent may perform the following sequence: Example 2: Tool Invocation Step 1: Select target tool legal_case_search Parameter = { query = “Acme’s tax dispute”, top_k = 5} Step 2: Select rel...
• In a typical scenario, users submit requests to the agent; the agent’s LLM decides which knowledge-based tool to invoke, translates the request into concrete tool arguments (e.g., search terms, filters, or top-k limits), and answers us- ing the retrieved in...

Abstract

Abstract unavailable from OpenAlex metadata.

Bullet Summary

• ChronosAttack introduces a delay-only adversarial scheduling attack targeting asynchronous LLM agents by manipulating the timing of authentic tool responses without altering their content.
• Bounded timing delays can reorder tool response arrivals, significantly changing the agent's final decisions despite all tool inputs being authentic and unmodified.
• The attack was evaluated on four varied LLM families (GPT-5.6 Sol, Gemini 3.6 Flash, DeepSeek V4 Flash, Claude Sonnet 4.6) showing diverse sensitivity to temporal ordering, with some models exhibiting strong targeted behavior shifts.
• Sequential agent states tend to increase vulnerability to timing attacks, but even stateless setups show notable sensitivity to the order in which observations are processed.
• This work highlights timing and tool-response scheduling as a novel system-level attack vector beyond traditional adversarial input or tool poisoning methods.