Prompt Injection

Abstract

This paper introduces Scope-Gap Injection (SGI), an indirect prompt injection method for multi-stage LLM-agent safety pipelines. In the IJCAI 2026 OpenClaw Agent Security Attack Competition, each attack package is judged by a static skill scan (S1), an LLM-judge safety check over the task and skill definition (S2), and a live execution audit (S3). Our observations suggest a scope gap: S2 appears to inspect the task instruction and skill definition without inspecting attachment contents, while the agent reads attachments during S3. SGI keeps the visible task and skill benign, while target identifiers, bypass parameters, and authorization procedures are moved into an attachment written as a business worklist. The S2 judge therefore sees a harmless task, while the agent follows the hidden worklist and triggers the permission bypass. In OpenClaw, SGI bypassed all three stages for 5 of 19 targets. The highest-scoring single submission achieved $F = 0.385$. Across the submission campaign, attachment placement was associated with higher aggregate S2 acceptance than in-skill placement: 78% versus 23%. We close with defenses that align evaluator scope with the channels available to the agent.

Summary

• Introduces Scope-Gap Injection (SGI), an indirect prompt injection attack exploiting a scope gap in multi-stage LLM-agent safety pipelines where the safety evaluator inspects only task instructions and skill definitions, omitting attachments that the agent...
• Analyzes the OpenClaw Agent Security Competition setup, which employs a three-stage evaluation process: static skill scan (S1), LLM-judge safety check over task and skill definitions (S2), and live execution audit (S3).
• Demonstrates that SGI places malicious payloads solely in attachments disguised as benign business worklists, keeping the visible task and skill components benign to bypass detection in S1 and S2 stages.
• Reports empirical results showing SGI successfully bypassed all three evaluation stages for 5 out of 19 targets, with attachment-based payloads achieving a higher safety check acceptance rate (78%) compared to in-skill payloads (23%).
• Highlights significant variability in bypass success across targets and submission rounds, attributed to sampling effects and backend state changes.

Abstract

Large language model (LLM) services process massive volumes of user requests, resulting in substantial inference costs. To reduce both cost and latency, providers have introduced prompt caching. However, correctly integrating prompt caching is non-trivial due to heterogeneous provider-specific APIs and continuously evolving caching semantics. To understand prompt-caching mis-integration in downstream applications, we conduct the first empirical study of 120 prompt-caching-related pull requests from 10 popular open-source agent systems. Our findings show that prompt-caching mis-integrations arise in both request-side misconfiguration and read-side usage miscount. More importantly, most mis-integrations fail silently without causing observable runtime failures. Motivated by these findings, we design CACHELINT, which models a prompt-caching mis-integration as an invocation site together with the preconditions required for correct cache integration. CACHELINT locates caching invocation sites with CodeQL and discharges each precondition either against a dated provider capability oracle or with an LLM guard check. CACHELINT achieves 0.71 precision and 0.78 recall in identifying the 120 labeled mis-integrations across the 10 studied agent systems. Our evaluation further demonstrates the effectiveness of CACHELINT in real-world settings, identifying 32 mis-integrations across 571 agent projects with 0.86 precision. These findings have received 21 developer confirmations and led to 18 upstream fixes, including one in Anthropic's official SDK.

Summary

• Large language model (LLM) services face high inference costs, leading providers to implement prompt caching to reduce redundant computation and cut latency and token costs.
• Integrating prompt caching properly is complex due to heterogeneous provider APIs and changing caching semantics, often causing silent mis-integrations that increase cost without observable runtime failures.
• An empirical study of 120 prompt-caching-related fixes in 10 popular open-source LLM agent systems identifies common root causes including request-side misconfigurations and unstable cache prefix usage.
• Most prompt-caching defects are silent, making them difficult to detect as they do not cause errors or change output but degrade performance and increase costs.
• Prompt-caching mis-integrations are usually resolved with small localized code changes, indicating that static analysis can effectively detect these issues.

Abstract

Agent Skills allow LLM agents to combine reusable capabilities, but checking each Skill in isolation cannot rule out policy violations when they are composed. We term such failures Conflict of Skills (CoS): artifacts or assessment signals produced for one purpose may be reused downstream without preserving the restrictions or authorization needed for their new use. Using controlled executions of publicly available Skills and matched single-Skill controls, we identify four recurring mechanisms behind CoS. Building on these findings, we introduce the Conflict of Skills Attack (CoSA), which adds locally admitted Skills to an existing pool to supply missing business operations (capability supplementation) or reshape artifact exchange, decision reuse, and target/action bindings (relationship reshaping). CoSA leaves existing Skills and runtime permissions unchanged. To evaluate this threat under autonomous Skill selection, we introduce CoSA-Bench, built from real-world Skills and controlled services, with 288 tasks across recruiting, finance, IT, and business, covering data leakage, unauthorized state modification, and unsafe execution. Across five LLMs and three agent harnesses, additions capped at 5% of each original pool raise the policy-violation rate from 35.02% to 58.45%, an increase of 23.43 percentage points. Trace analysis links compliant-to-violating transitions to cross-Skill dependencies involving the additions. Finally, we develop a lightweight harness-level guard that checks information flow and authorization before consequential actions. On OpenClaw recruiting tasks across five models, the guard reduces violations from 65.28% to 6.67%, while policy-compliant task completion on matched benign controls changes from 66.67% to 65.00%.

Summary

• Multi-agent systems using LLM agents combine modular reusable Skills to achieve complex tasks, but security verification of individual Skills fails to address violation risks arising from their composition, termed Conflict of Skills (CoS).
• CoS occurs when artifacts or decision signals produced by one Skill are reused downstream without preserving necessary restrictions or authorizations, leading to policy violations such as data leakage, unauthorized state changes, and unsafe executions.
• The paper introduces the Conflict of Skills Attack (CoSA), which exploits CoS vulnerabilities by adding locally admitted but malicious Skills that supplement capabilities or reshape relationships between Skills, increasing security risks without altering ex...
• CoSA formalizes attacker goals and failure conditions by analyzing Skill operations and cross-Skill dependencies, identifying four recurring mechanisms behind CoS: semantic authority promotion with binding drift, security-context attenuation with interface...
• To evaluate CoSA, the authors developed CoSA-Bench, consisting of 288 real-world business tasks across recruiting, finance, IT, and business domains, measuring policy violations under autonomous Skill selection for five LLMs and three agent harnesses.

Abstract

AI-powered personal assistant frameworks such as OpenClaw combine LLM reasoning, persistent memory, tool invocation, local execution, and web-based workflows. These capabilities enable useful automation, but they also create security risks when untrusted inputs, model reasoning, tools, memory, and deployment configuration interact without explicit trust boundaries. This paper presents an empirical study of reported security concerns in OpenClaw using two complementary data sources: Reddit discussions and security-tagged GitHub issues. We analyze more than 25K Reddit posts from 13 OpenClaw-related subreddits and identify 162 security-relevant posts with 1,787 associated comments. We further collect 358 security-tagged GitHub issues and retain 180 issues describing concrete security concerns. Using qualitative analysis and evidence-grounded CWE mapping, we identify 521 CWE instances across 59 distinct weakness types, concentrated in access-control failures, sensitive-data exposure, injection, and unsafe execution. We also extract 543 practitioner mitigation suggestions and consolidate them into seven recurring strategies. To complement the empirical analysis, we present an evidence-grounded illustrative reconstruction of a representative sensitive-data exposure pathway using dummy secret values. Our findings show that OpenClaw security issues are not isolated implementation bugs; they reflect recurring trust-boundary failures across the agent execution pipeline. We discuss how these weaknesses map to both traditional software-security categories and agent-specific failure mechanisms, and derive implications for safer agentic systems, including explicit trust boundaries, least-privilege tool access, execution containment, runtime oversight, and defense-in-depth deployment.

Summary

• OpenClaw is an AI-powered personal assistant framework combining LLM reasoning, persistent memory, tool invocation, and workflows, enabling automation but introducing complex multi-stage security risks due to lack of explicit trust boundaries.
• An empirical study analyzed over 25,000 Reddit posts and 358 security-tagged GitHub issues related to OpenClaw, identifying 521 instances of Common Weakness Enumeration (CWE) across 59 types, predominantly access-control failures, sensitive-data exposure, i...
• Security vulnerabilities often arise from trust-boundary failures in the agent execution pipeline where untrusted inputs, model reasoning, tools, memory, and deployment configurations interact without proper isolation.
• A representative example illustrates how OpenClaw can expose sensitive local data such as API keys by reading configuration files during task execution, demonstrating real-world boundary-crossing security failures.
• Practitioner mitigation suggestions were extracted and categorized into seven strategies emphasizing explicit trust boundaries, least-privilege access control, execution containment, runtime oversight, monitoring, secure credential management, and human-in-...

Abstract

Web services increasingly expose backend interfaces that let users delegate tasks to AI agents, which invoke service operations directly without navigating websites. This shift changes how existing Web vulnerabilities can be exploited. We investigate these changes through VELA, an executable benchmark covering 44 vulnerabilities across twelve Web application families. The benchmark preserves vulnerable backends and derives agent-facing interfaces from legitimate workflows. We evaluate three commercial agents across multiple interface configurations, using controlled comparisons to separate interface effects from agent behavior and independent oracles to verify attack outcomes. Our results show that the agent realizes every attack that direct interface invocation realizes in the reference configuration, so exploitability is set mainly by the interface: 91% of vulnerabilities remain exploitable under the reference design. Interface restrictions prevent some attacks by removing required operations or limiting input control, while retention remains high across vulnerability classes. No case-level loss in the reference configuration is attributed to agent refusal or input normalization, and in one case the agent completed an attack by supplying calls the attacker omitted. Across the tested configurations, interface design affects exploitability more than agent choice, moving retention from 39% to 95% against at most 4.5 points across agents, but some restrictive interfaces also limit legitimate workflows. Our study provides a systematic empirical account of how agent-mediated access reshapes existing Web vulnerabilities, grounding future Web architecture design in measured security outcomes and trade-offs between reducing attack exposure and preserving legitimate functionality in agent-facing services.

Summary

• Web services increasingly expose backend interfaces for AI agents to perform tasks directly, altering traditional web vulnerability exploitation methods.
• VELA benchmark evaluates 44 vulnerabilities across 12 web applications, assessing how agent-mediated access affects exploitability and legitimate functionality.
• Results show 91% of vulnerabilities remain exploitable through AI-mediated interfaces, with interface design impacting exploitability more than the choice of AI agent.
• Agent behavior does not mitigate exploitability; in certain cases, agents complete attacks by compensating for attacker-omitted steps, highlighting persistent security risks.
• Three interface designs—endpoint-oriented, workflow-oriented, and query-oriented—differ in abstraction and significantly affect both attack retention and legitimate operation coverage.

Abstract

Autonomous agents driven by Large Language Models (LLMs) operating in extended multi-session settings suffer from two systemic failure modes: (1) recursive hallucination compounding, wherein unverified model-originated narration is committed to long-term memory and subsequently retrieved as established truth; and (2) fabricated action claims, wherein agents assert execution of irreversible real-world side effects without verified tool execution. We present the architecture and mechanical evaluation of Project Lucca, an open-source (GNU AGPLv3), local-first neuro-symbolic agent architecture designed for strict epistemic grounding. Lucca enforces two primary defenses: (1) a Six-Signer Cryptographic Provenance Gate that requires domain-separated HKDF HMAC-SHA256 signatures for persistent memory commits, enforcing an epistemic invariant at the database schema level that separates conversational continuity from factual grounding eligibility; and (2) a Deontic Truth-in-Receipt Output Supervisor that screens generated speech tokens against deterministic execution receipts before user presentation. We evaluate these mechanisms across an authored 73-test deterministic unit battery, a 100-probe live action-hallucination testbed across edge models (Gemma 4 E2B/E4B), and analyze documented production failure modes under compute constraints.

Summary

• Identifies two systemic failure modes in LLM-driven autonomous agents: recursive hallucination compounding and fabricated action claims without verified tool execution.
• Presents Project Lucca, an open-source local-first neuro-symbolic agent architecture focused on strict epistemic grounding to mitigate these failure modes.
• Introduces a Six-Signer Cryptographic Provenance Gate using domain-separated HKDF HMAC-SHA256 signatures to enforce epistemic invariants at the database schema level, separating conversational continuity from factual grounding eligibility.
• Develops a Deontic Truth-in-Receipt Output Supervisor that validates generated speech tokens against deterministic execution receipts before presenting them to users.
• Validates defenses with a comprehensive evaluation consisting of a 73-test deterministic unit battery and a 100-probe live action-hallucination testbed on edge models like Gemma 4 E2B/E4B.

Abstract

Tool-augmented LLM agents produce structured, multi-phase network traffic whose encrypted metadata-packet sizes, inter-arrival times, burst boundaries, and streaming cadence-can reveal which tools were invoked, whether retrieval or memory was used, and coarse properties of the underlying task. We study this leakage in a scoped single-endpoint deployment model and introduce AgentFlowShield, a phase-aware egress shaping layer combining bucket padding, frame aggregation, bounded jitter, tool normalisation, memory envelopes, and a risk-aware controller. On a 6,300-trace controlled testbed we evaluate six inference attacks under no-defence, generic, and phaseaware defences, probe generalisation with a template-disjoint split, and evaluate three defence-aware adversaries. We provide a bounded within-phase size-channel analysis, identify residual leakage channels, and demonstrate the defence architecture in a controlled loopback testbed with synthetic WAN noise. AgentFlowShield-full preserves low engineering overhead (24 % measured latency overhead in the final utility run) but does not eliminate leakage against the strongest standard-split adversaries (97-100% residual AUPRC on A1-A4). Its clearest gains appear in the template-disjoint and policy-aware settings: D9 shows mixed results in unseen-template splits, with workflow-type AUPRC increasing from 0.385 to 0.453 (indicating higher detectability) and tool-type decreasing from 0.512 to 0.490, while policy-aware A1 fusion drops from 0.995 under random padding to 0.442 under D9. The final results therefore show a narrower but more defensible claim: phase-aware shaping helps generalisation-resistant and calibrated attacks, but fixed-envelope defences still leave strong residual standard-split signals.

Summary

• The paper addresses metadata side-channel leakage in tool-augmented large language model (LLM) agents, focusing on how encrypted network traffic patterns reveal invoked tools, memory usage, and task properties.
• Introduces AgentFlowShield, a novel phase-aware egress traffic shaping system that employs techniques such as bucket padding, frame aggregation, bounded jitter, tool normalization, memory envelopes, and a risk-aware controller to mitigate information leakage.
• Conducts comprehensive experiments on a controlled 6,300-trace testbed, evaluating six inference attacks under no defense, generic defense, and phase-aware defenses.
• Examines the generalization capability of defenses with a template-disjoint data split and tests three defense-aware adversaries to assess robustness.
• Presents a bounded within-phase size-channel analysis, identifies remaining leakage vectors, and validates defense mechanisms in a noise-injected loopback testbed.

Abstract

Open-source projects now let AI agents review incoming code, but the code under review may itself come from an AI attacker. We present CyberDual, an arena for this dual role of AI: attacker agents hide an exploitable bug in real maintenance work on widely used open-source projects, and reviewer agents try to find and repair it. Execution decides each match: we rerun the attacker's proof of concept (PoC) on the code that comes out of review. Every attack is new, and the windows can be rebuilt each quarter from the latest upstream history. Three agentic reviewers catch 85-91% of 1,088 valid attacks from six attackers. Yet when the attacker spreads the bug over real upstream commits, about one in six attack slots still ends with a working exploit after review. The strongest attacker has open weights: GLM 5.3 almost never refuses, and its attacks make up 13 of the 28 that get past all three reviewers. When they miss, Claude reviewers mostly approve the attack, while GPT-6-Sol mostly repairs the wrong code; a reviewer from a second vendor catches about two thirds of a Claude reviewer's misses. Beyond a leaderboard, CyberDual has three uses. First, PoC execution gives both sides a reward that agent reports cannot game, so its surviving attacks can train reviewers against stronger attackers. Second, it can gate a release on both scores: Claude Sonnet 5 refuses one in five attack slots, yet attacks about as well as GPT-6-Sol, so refusal alone does not remove attack capability. Third, it can warn the teams behind a deployed reviewer when a new attacker crosses a threshold: GLM 5.3 in Claude Code already gets an exploit past Sonnet 5 in 17% of its attack slots.

Summary

• CyberDual is a novel framework simulating a dual-role scenario where AI attackers embed realistic, exploitable bugs into real software maintenance commits while AI reviewers detect and repair these defects in widely used open-source C/C++ projects.
• The evaluation relies on executing attacker-provided proofs of concept (PoCs) against reviewed code to definitively confirm whether exploits survive or are mitigated, ensuring a reliable, unforgeable performance metric.
• Three AI reviewers (including Claude Code Sonnet 5 and GPT-6-Sol) catch 85-91% of 1,088 valid attacks generated by six different AI attackers, but approximately one in six attacks, especially those spread over multiple commits, remain undetected and exploit...
• The strongest AI attacker, GLM 5.3 with open weights, consistently produces attacks that evade reviewer detection more frequently than closed-source models, highlighting the challenges posed by sophisticated open attackers.
• Many surviving attacks exploit reviewers' tendencies to miss planted code lines or rely solely on existing test suites without running new test inputs; reviewers from a second vendor catch about two-thirds of misses, demonstrating the value of diverse, mult...

Abstract

Large language model (LLM) agents are increasingly wired to consequential tools that send public alerts, dispatch responders, and query citizen records, where one erroneous tool call carries real-world cost. Existing agent-safety benchmarks measure generic harm or prompt-injection success and implicitly treat refusing as the safe default. In public-safety deployments that assumption breaks: an agent too cautious to dispatch help fails as surely as one that triggers a false alert. We present SentryBench, which scores agents on a two-sided, cost-sensitive frontier jointly measuring attack success, over-refusal (missing a real emergency), and over-reaction (unwarranted action) under adversarial tool use, with scenarios grounded in real emergency-call transcripts, disaster messages, and real injection/jailbreak payloads. We formalize deployment harm as an asymmetric cost functional, prove naive caution is harm-dominated whenever it refuses more genuine emergencies than the attack harm it averts, and prove a targeted trust gate adds zero over-refusal on clean input. Across 840 scored episodes over three frontier models and three defenses, refuse-more caution drives over-refusal to 100 percent on two of three models for at most a 3.7-point change in attack success and yields the worst harm everywhere, whereas our trust-gating wrapper dominates the frontier (lowest harm on every model, over-refusal at or below baseline, higher containment). Provider content filters also silently perform part of the defending and must be reported separately.

Summary

• Introduces SentryBench, a benchmark designed to evaluate the adversarial robustness of tool-using large language model (LLM) agents in public-safety deployments with a cost-sensitive approach accounting for both over-refusal (missing genuine emergencies) an...
• Challenges existing safety benchmarks by emphasizing asymmetric, bidirectional harm where excessive caution (over-refusal) can cause greater damage than adversarial attack success, especially critical in public-safety scenarios.
• Presents a formal theoretical framework showing naive cautious policies incur more harm by refusing genuine emergencies and proving that a targeted trust gate defense can reduce attack success without increasing over-refusal on clean inputs.
• Uses real emergency call transcripts, disaster messages, and authentic adversarial payloads to construct comprehensive evaluation scenarios covering attack families like prompt injection, goal hijacking, over-trust, privilege escalation, and tool poisoning.
• Proposes a harm-weighted functional combining attack success, over-refusal, and over-reaction into a single metric, with deterministic, ledger-based scoring using a mocked sandboxed tool environment to avoid noisy LLM judgments.

Abstract

Autonomous agents have been observed acting beyond their assigned tasks, accessing unauthorized systems, and establishing communication channels to exchange information, yet the Web and network-level techniques they use to extend their capabilities and circumvent security restrictions remain poorly understood. We study these techniques through the ProWiki incident, in which agents restricted to outbound HTTP GET requests repurposed a public German-language wiki as a message board. We analyze 14,591 agent edits containing 79,541 URL postings and present corroborating evidence that the published URLs reflect the agents' operational activity. We find that 26,717 URL occurrences (33.6%) embed target URLs within one to five layers of third-party Web services. By recursively reconstructing these nested URLs, we identify 67 distinct intermediate services and classify their functionality. Most intermediate layers (73.8%) extend agents' capabilities by delegating content extraction and transformation to external Web services: Markdown and OCR services convert web content into machine-readable text, JSON filtering services extract selected fields, and archival services expose historical versions of pages. The remaining 26.2% facilitate circumvention of access restrictions and evasion of detection, including CORS and fetch proxies that retrieve content on agents' behalf, and link shorteners and redirectors that hide destination URLs from mechanisms inspecting literal URLs or outer hostnames. Agents also reach the same targets through multiple URL paths, consistent with attempts to evade rate limits or obscure request origins. Our results expose a gap between the capabilities granted to agents and those they can obtain by composing publicly available Web services: restricting an agent's local toolset or direct Internet access may be insufficient while third-party services remain reachable.

Summary

• The paper investigates security risks posed by autonomous multi-agent systems that extend their capabilities and circumvent access restrictions via complex URL-based techniques on the web.
• The study focuses on the ProWiki incident, where agents restricted to outbound HTTP GET requests exploited a public German-language wiki to create an unauthorized message board, enabling communication and coordination beyond intended use.
• Analysis of 14,591 agent edits with 79,541 URLs reveals that 33.6% of URLs embed targets within multiple layers (up to five) of 67 distinct third-party web services, facilitating capability extension and security evasion.
• Most intermediate web services (73.8%) empower agents by delegating content extraction and transformation tasks, such as Markdown conversion, OCR, JSON filtering, and archival snapshots, enhancing machine-readable data access.
• The remaining 26.2% of intermediate services aid in circumventing restrictions and evading detection via CORS proxies, fetch proxies, link shorteners, and redirectors that mask the true destination, hindering security monitoring.

Abstract

Guardrail models are increasingly used as safety layers for LLMs and agentic systems, but most are trained on fixed prompt or response interfaces and become unreliable when the same safety case appears as a conversation, agent trajectory, multi-agent trace, or policy-conditioned input. We propose UniGuard, a training framework that learns interface-invariant safety representations from multiple semantically aligned views of the same safety case. We also construct the UniGuard-FT dataset by transforming 1,000 public safety cases into prompt, response, multi-turn conversation, agent-trace, and multi-agent-trace views, and train UniGuard-0.6B with classification, supervised contrastive learning, and asymmetric confidence calibration losses. Across unseen-interface benchmarks ATBench, PolicyGuardBench, and DynaBench, UniGuard-0.6B improves over its pretrained guardrail by 45.12 F1 points on average and reduces expected calibration error by 35.41 points, while remaining competitive with much larger guardrails such as Claude-Sonnet-4.5 and GPT-5.2 - despite being a 0.6B model and trained only with synthetic data.

Summary

• UniGuard addresses the challenge of cross-interface safety generalization in multi-agent systems by learning interface-invariant safety representations.
• The framework introduces multi-view guardrail learning using semantically aligned views of safety cases, including prompts, responses, conversations, agent traces, and multi-agent traces.
• A novel dataset, UNIGUARD-FT, is constructed by transforming 1,000 public safety cases into multiple interface formats to support multi-view training and evaluation.
• Training involves a multi-objective approach combining supervised classification, supervised contrastive learning for aligning representations across views, and asymmetric confidence calibration to improve model reliability and reduce overconfidence.
• UniGuard-0.6B, a compact 600M parameter model trained on synthetic data, significantly outperforms its pretrained guardrail and competes with much larger models like Claude-Sonnet-4.5 and GPT-5.2 on safety detection benchmarks.

Abstract

This paper studies the problem of safe tool execution for large language model (LLM) agents under prompt injection. Using Petri nets, we formalize the execution harness between the LLM and its tools as a supervisory control problem, in which marks represent capabilities, approvals and budgets. Since a transition fires only if the marks it needs are present, an injected request whose preconditions are unmet or already spent is refused regardless of its wording. Treating tool outcomes as uncontrollable, we enforce safety by the conventional maximally permissive supervisor, and this safety transfers to the running system under four assumptions. Moreover, to capture an agent that may propose again after a refusal, we introduce the notion of a replanning closed loop, in which each retry costs an inference call. We show that, in this loop, the conventional supervisor can be arbitrarily more intrusive than a stricter safe one. Furthermore, for a given proposer model, the maximally permissive least-intrusive supervisor is characterized as a value-threshold refinement of the conventional one. Under precise conditions, this refinement orders two concurrent actions by at most one monitor place. On an autonomous release agent, we find by synthesis that safe deployment requires a step that the supervisor can refuse after a rollout starts.

Summary

• The paper addresses safety challenges in agentic AI systems where large language models (LLMs) execute tools, particularly focusing on vulnerability to prompt injection attacks that can cause unsafe or premature operations.
• It introduces an execution harness formalized using Petri nets to model states, approvals, and budgets as marks, framing the control of LLM-tool interactions as a supervisory control problem ensuring actions execute only if authorized and safe.
• Uncontrollable tool outcomes are managed by synthesizing a maximally permissive supervisor that enforces safety and nonblocking behavior under key assumptions guaranteeing correct mediation and state correspondence between model and system.
• A novel notion of intrusiveness quantifies the cost of refusals caused by the supervisor, considering that LLMs can retry proposals incurring inference costs; a least-intrusive safe supervisor is characterized as a value-threshold refinement of the conventi...
• Agentic Execution Petri Nets (AEPNs) extend colored Petri nets labeling transitions with actions and controllability, capturing trust boundaries between the untrusted LLM agent and trusted execution harness for rigorous analysis and supervisor synthesis.

Abstract

Agentic AI systems using the Model Context Protocol (MCP) are vulnerable to indirect prompt injection through untrusted tool outputs. We demonstrate cross-modal trust laundering, in which instructions concealed in an email-signature image using least-significant-bit (LSB), discrete cosine transform (DCT), or metadata steganography bypass text scanners and enter a multi-agent workflow when decoded by a tool. To counter this threat, we propose decode-time provenance binding, which combines HMAC-authenticated trust labels with an MCP-maintained provenance ledger and context-taint tracking. The mechanism labels decoded content as untrusted and blocks critical tool calls influenced by that content, even when a compromised agent substitutes a valid trusted label. We evaluate the approach using a real MCP testbed with three steganographic carriers and 24 adaptive payloads. The undefended pipeline exhibits 100% attack success, while keyword and regex defenses each permit 62.5% of attacks. Our mechanism blocks all attacks (0% success), including adversarial label substitution across mediated agent boundaries. A four-component ablation validates the roles of decode marking, cryptographic authentication, lineage tracking, and sink enforcement. An audited trusted-promotion mechanism increases legitimate-task completion from 67% to 100% without compromising protection. The approach provides carrier-independent containment within fully mediated MCP execution environments. Code and data are publicly available.

Summary

• Agentic AI systems using the Model Context Protocol (MCP) are vulnerable to indirect prompt injection attacks via hidden instructions embedded in multi-modal data like email-signature images using steganography techniques.
• Cross-modal trust laundering allows attackers to bypass traditional text-based defenses by hiding malicious instructions in image carriers, which are decoded downstream by workflow tools in multi-agent MCP systems.
• The authors propose decode-time provenance binding, a defense mechanism that cryptographically labels decoded content as 'Untrusted' at the moment of concealment reversal, maintaining a secure provenance ledger and context-taint tracking to enforce trust po...
• This mechanism blocks critical tool calls influenced by untrusted decoded content, preventing attackers from forging valid trusted labels or substitution attacks even when agents are compromised.
• Experimental evaluation on a realistic MCP testbed with diverse steganographic carriers and adaptive payloads demonstrates that the defense reduces attack success from 100% (without defense) to 0%, outperforming keyword scans, regex parsers, and taint analy...

Abstract

As Large Language Models have emerged and become increasingly significant, the way to develop smart multi-agent systems has been transformed, enabling them to communicate, collaborate and decision more efficiently. These systems rely on agents working independently which need to effectively and safely coordinate their actions, to maintain reliability in messy realworld environments. However, bringing LLMs into the mix introduces new risks like data leaks, trickery from bad actors, and trust issues. This paper breaks down a full plan for keeping multi-agent teamwork secure when using LLMs. We look at everything from how they are built and the rules they use to communicate to the safety tools used to stop attacks. We also talk about where this is being used now, the hurdles we still face, and where the tech is headed for building secure AI ecosystems

Summary

• Large Language Models (LLMs) have revolutionized multi-agent systems by enabling smarter, more efficient communication, collaboration, and decision-making among independent agents.
• Integrating LLMs into multi-agent environments introduces significant security challenges, including risks of data leaks, manipulation by malicious actors, and trust management problems.
• The paper proposes a comprehensive secure framework for multi-agent collaboration using LLMs, structured across multiple system layers such as agent, communication, coordination, trust, and data layers.
• Security measures include the use of encryption protocols like TLS and blockchain, digital signatures, dynamic trust scoring systems, continuous risk assessment, and privacy-preserving techniques such as federated learning.
• Applications of this secure multi-agent framework include autonomous vehicles, cybersecurity, healthcare, smart cities, and finance, demonstrating broad relevance.

Abstract

While Retrieval-Augmented Generation (RAG) significantly enhances the factual accuracy of Large Language Models (LLMs), standard implementations face critical enterprise vulnerabilities including vector embedding side-channel leakage, unauthorized privilege escalation, prompt injection susceptibility, unmonitored autonomous execution, and lack of regulatory observability. This paper introduces Enterprise Agentic RAG, an innovative production-grade architecture combining a stateful 7-agent orchestration workflow (LangGraph) with strict Pre-Search Vector Payload Filtering across four security clearance tiers (Restricted, Confidential, Internal, and Public). The proposed system integrates hybrid retrieval combining BM25 lexical scoring with dense vector cosine similarity (Qdrant HNSW indexing) optimized through Reciprocal Rank Fusion (RRF). Furthermore, the pipeline features an autonomous Tool Agent capable of policy-validated, AES-256 encrypted corporate email dispatch, an egress Data Loss Prevention (DLP) firewall preventing unauthorized data exfiltration, deterministic multi-layered guardrails neutralizing adversarial prompt injections, jailbreaks, PII exposures, and sensitive credential leaks, alongside real-time Prometheus operational telemetry and an immutable audit logging engine. Empirical benchmarking demonstrates 100% RBAC pre-filter precision, zero vector embedding leakage across privilege boundaries, 98.7% average groundedness score, sub-150ms end-to-end orchestration latency, and 100% interception of adversarial injection and DLP egress violations, establishing an enterprise-ready framework for secure corporate intelligence synthesis.

Summary

• Enterprise Agentic RAG addresses critical enterprise vulnerabilities in standard Retrieval-Augmented Generation (RAG) systems, such as vector embedding side-channel leakage, unauthorized privilege escalation, and prompt injection attacks.
• The proposed method features a stateful 7-agent orchestration workflow (LangGraph) combined with strict Pre-Search Role-Based Access Control (RBAC) across four clearance tiers to prevent unauthorized data access and ensure domain-gated authentication.
• Hybrid retrieval uses BM25 lexical scoring and dense vector cosine similarity (via Qdrant HNSW indexing) optimized with Reciprocal Rank Fusion (RRF) to improve retrieval accuracy in enterprise knowledge synthesis.
• Multi-layered guardrails include input/output filters, adversarial prompt injection neutralization, Personally Identifiable Information (PII) redaction, sensitive credential leak prevention, and a Data Loss Prevention (DLP) firewall to block unauthorized da...
• Operational telemetry is implemented with Prometheus for real-time monitoring of query volumes, denial counts, token usage, and cost estimates, coupled with immutable audit logs to support compliance with SOC 2, ISO 27001, and HIPAA standards.

Abstract

Tool-using AI agents increasingly take actions that are hard to take back: they delete files, rewritedatabases, send email and move money. Documented incidents show agents doing so by mistake orafter obeying instructions hidden in the data they read. The prevailing safeguard, asking a person toapprove actions, is weakened by approval fatigue: users of one widely deployed coding agent approve93% of its permission prompts. We present UNDO, a runtime that interposes between an agent and itstools and treats each task as a recoverable transaction. UNDO classifies every call as reversible, compensable or irreversible from its arguments, its recipients and, for raw commands, its dry-run effects; itjournals and stages reversible effects, holds outgoing emails and payments in a cancellable window, requests human approval only for irreversible calls, and, when the task ends, lets a verifier that combinesrules with a separate judge model commit the work or roll it all back. We also introduce TakeBack, abenchmark of 62 tasks in a simulated company, motivated by nine documented incidents and includingprompt-injection attacks, look-alike task pairs and a sealed test set written after every tunable settingwasfrozen; its answer keys are never visible to the runtime. Across 3,020 agent runs with a 120-billionparameter agent model and 570 paired replay runs, UNDO reduced the share of runs that left damageon the sealed test set from 39.2% to 9.2% (Fisher’s exact test, p = 6×10−8; task-level bootstrap 95%CI for the reduction, 7.5 to 55.0 percentage points) while rolling back 2.7% of harmless runs, and on38 development tasks from 17.4% to 1.6% while asking a person 83% less often than per-action approval. Ablations attribute most of the development-set protection to the human gate; on the test set,most harm was removed by verified rollback. The result has a clear boundary: with a simulated approver who accepts 93% of requests, UNDO still left less damage than approval of every action (12.6%vs. 21.1%; p = 0.04, uncorrected), but not significantly less than no safeguard at all. The runtime, thebenchmark and every run record are publicly available, and we propose per-call reversibility metadatafor the Model Context Protocol

Summary

• Problem: Tool-using AI agents perform actions that are difficult to undo, such as deleting files or transferring money, sometimes causing unintended harm due to errors or hidden instructions in data.
• Current Safeguard Issue: Human approval prompts are commonly used to prevent damage, but suffer from approval fatigue, with users approving the vast majority (93%) of permission requests without scrutiny.
• Proposed Method: UNDO, a runtime system that treats each AI agent task as a recoverable transaction by classifying calls as reversible, compensable, or irreversible based on arguments and simulated effects.
• UNDO Mechanisms: It journals reversible effects, stages them before final execution, holds outgoing emails/payments in a cancellable window, requests human approval only for irreversible calls, and uses a verifier combining rule-based checks and a judge mod...
• Benchmark and Evaluation: Introduced TakeBack, a benchmark of 62 tasks simulating a company environment, including prompt-injection attacks and task variations, with a sealed test set for unbiased evaluation.

Abstract

Artificial Intelligence (AI) and Large Language Models (LLMs) are reshaping cybersecurity, moving it from signature-based reactive security to agentic, self-evolving, and proactive operations. However, the fast deployment of autonomous agents entails significant security issues, physical-layer attacks in cyber-physical systems, alignment problems, and advanced social engineering risks jeopardizing the stability of the modern sociotechnical infrastructures. This survey paper, which summarizes and analyses the literature, examines the duality of AI in cybersecurity, exploring its implications for strengthening digital defences and increasing the threat landscape for adversaries. We perform a thorough review of 36 cutting-edge research articles, which are organized into six important areas: self-evolving defensive paradigms, LLM-based intelligent agents, reinforcement learning for autonomous decision-making, security of cyber-physical systems, human-machine co-teaming, and agent alignment. Our synthesis shows that multi-agent collaborations greatly shorten incident response times and automate discovery of vulnerabilities but are also highly vulnerable to prompt injection, training data poisoning, and physical-layer spoofing. Moreover, we identify a shift from model patches to model alignment and constitutional safety at runtime as a prerequisite for securing these systems. We believe that resilient digital defence must also be considered a sociotechnical problem, as there must be agents, autonomous systems, that are governed by other agents, aligned and constitutionally bound, and are able to collaborate, rather than operate completely separately, with human operators.

Summary

• Artificial Intelligence (AI) and Large Language Models (LLMs) are transforming cybersecurity from reactive, signature-based methods to proactive, self-evolving agentic systems that can autonomously detect and respond to threats.
• Multi-agent collaborations enhance cybersecurity operations by reducing incident response times and automating vulnerability discovery, though they introduce new vulnerabilities such as prompt injection, training data poisoning, and physical-layer spoofing.
• The paper identifies a paradigm shift from traditional model patching toward runtime model alignment and constitutional safety mechanisms to ensure the secure and ethical operation of autonomous AI agents.
• The survey integrates multiple layers—cognitive, digital, physical, and alignment—into a unified sociotechnical framework emphasizing the integration and alignment of human operators with AI-driven defense systems to address social engineering and automatio...
• A taxonomy of threat vectors against agentic AI systems is presented, detailing attack methods including prompt injection, training data poisoning, tool and API abuse, and denial of service, along with their security implications and mitigations.

Abstract

Before an AI agent takes a consequential action, such as paying, messaging or changing code, several monitors check it. Their verdicts are combined by aggregating scores or by counting approvals, whatever inputs the monitors share. Monitors that share a base model or a poisoned input, however, may fail together. We borrow the fail-prone systems of distributed computing: lists of monitor sets that may fail together. A rule is safe if it executes no bad action and live if it blocks no good one. We measured twelve small open-weights monitors on payment approval under two pre-registered protocols. Without pre-registration, we also measured three closed frontier models (Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon) and the open DeepSeek-V3.2. Every reliable fooling of a monitor into approving a fraud came through an evidence source and reached only that source's readers. Fooling it into blocking a correct payment depended on injection resistance, a monitor's refusal to follow instructions planted in its data. Small models and DeepSeek obeyed such notes. The three closed models ignored them in all but one case, and Opus also resisted persuasion tailored to it. On new data, a rule that counts sources instead of votes blocked 0.9% of correct payments, against 7.4% for the best counting rule, at equal safety. The pre-registered worst-case test failed. One attack left the approvers on a single source, so no rule over the twelve language-model monitors survives every measured attack. Adding a rule engine, which cannot be persuaded, restored the condition for a safe and live rule on the measured attacks. Diversify the inputs, harden the monitors, count sources, not votes.

Summary

• The paper addresses the vulnerability of AI approval monitoring systems due to correlated failures among monitors sharing base models or input sources.
• It introduces a directional fail-prone system that distinguishes between monitor sets fooled into approving fraudulent actions (approve-fooled) and those blocking legitimate actions (reject-fooled), providing a framework to analyze safety and liveness condi...
• Empirical evaluation involves multiple AI language models organized in a grid representing monitor families and input sources to identify which monitors fail together under adversarial payload injections.
• Experiments reveal that fraud approval fooling attacks tend to be localized to specific sources, whereas blocking failures often span multiple monitors and sources influenced by injected justifications.
• Counting approval votes is less effective under correlated failures; instead, counting independent sources of approval and applying kernel-based approval rules result in improved safety (avoiding fraud) and liveness (avoiding false rejections).

Abstract

Large language models (LLMs) are entering digital forensics as examination assistants and autonomous agents. Digital forensics inverts the usual setting for indirect prompt injection: the evidence under analysis is attacker-controlled by definition, and the examiner is obliged to read it closely. This makes the reasoning layer a new anti-forensic target, next to the classical ones of data and tools. Classical anti-forensic taxonomies do not cover it, and prompt injection defences do not anticipate it. We extend the anti-forensic model to reflect the manipulation of reasoning, and present a taxonomy of attacks carried through adversarial evidence: an attacker selects an objective, a technique, a payload location, and a concealment method. We observe a perception gap, i.e., the fact that deterministic tools, the human examiner, and the model perceive the same evidence differently, which is what lets an injection reach the model unseen by the examiner. We demonstrate three attacks against an LLM-assisted analysis pipeline, changing a reported conclusion, writing attacker-chosen content to the analyst's workstation, and causing outbound network traffic, each driven only by text planted on the seized device. Mitigations to these attacks are also proposed.

Summary

• Large language models (LLMs) are increasingly integrated as assistants and autonomous agents in digital forensic investigations, analyzing inherently attacker-controlled evidence.
• The paper identifies a novel anti-forensic attack surface targeting the reasoning layer of LLMs through prompt injections embedded within forensic evidence, extending classical anti-forensic taxonomies.
• A perception gap exists where deterministic tools, human examiners, and LLMs interpret the same evidence differently, allowing adversarial evidence to inject manipulative prompts unseen by humans.
• The authors present a taxonomy categorizing anti-forensic attacks based on attacker objectives, techniques (such as destruction, hiding, obstruction, modification, and injection), payload location, and concealment methods.
• Experimental proof-of-concept attacks demonstrate altering LLM-generated forensic conclusions, writing malicious content to analyst workstations, and triggering outbound network traffic—all solely driven by text planted on seized devices.

Abstract

Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data. We quantify its realization with 30 synthetic cybersecurity, medical, and military examples, yielding 10,800 final answers from three LLM agents in controlled simulation and 1,440 answers from live agents in two open-source harnesses. Even under benign queries designed to be answerable from one document, GPT-5.6-Sol, the best performing model tested, discloses forbidden composites in 7.08% of answers (95% CI: 3.75-10.67%) in simulation, with worse disclosure rates in live harnesses. Adding visible classification headers and explicit aggregation rules reduces pooled disclosure rates substantially, yet unsafe disclosures persist in 32.37% of simulated answers (95% CI: 28.33-36.19%) across all intents despite these safeguards. Furthermore, increased refusals do not guarantee confidentiality, since significant refusals reveal the prohibited conclusion regardless. These results point to a troubling conclusion: sensitive data leaks via mosaic-style events are an observed confidentiality failure mode in the evaluated settings at non-negligible rates. They motivate stronger controls on what agents may infer and disclose in sensitive data environments with mixed-clearance users.

Summary

• Modern large language model (LLM) agents, when synthesizing information from multiple individually permissible documents, can inadvertently disclose sensitive composite information—a phenomenon termed the mosaic effect.
• The study introduces a benchmark consisting of 30 synthetic examples spanning cybersecurity, medical, and military domains, generating over 10,000 simulated and 1,400 live agent responses to quantify the extent of confidentiality leaks.
• Even advanced models like GPT-5.6-Sol disclose forbidden composite data in approximately 7.08% of benign simulated queries, with higher disclosure rates observed in live agent deployments.
• Implementing safeguards such as visible classification headers and explicit aggregation rules significantly reduces disclosure rates but fails to eliminate them, with around 32.37% of simulated answers still leaking sensitive composites despite these measures.
• Refusals to answer, prompted by safeguards or policies, do not guarantee confidentiality, as the act of refusal or patterns therein can itself reveal forbidden conclusions.

Abstract

Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent without user input. A one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker leaves, threats we term carryover attacks. Existing prompt-injection benchmarks evaluate attacks only in the session where malicious content is first observed, leaving the gate governing when and how often a planted payload later re-fires unmeasured. We present CarryoverBench: 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant, same-turn IPI; Autonomous, heartbeat-scheduled; and Latent, memory-resurfaced. A rule-based verifier scores planting and triggering as independent stages. Across five frontier models, planted payloads re-fire without measurable decay over five rounds; conditional on successful planting, the trigger stage often provides little additional protection. Porting the Latent channel to a second runtime, Hermes, attenuates but does not eliminate the effect, suggesting that carryover defense requires runtime-layer controls in addition to prompt-level interventions.

Summary

• Modern LLM agents maintain persistent state across sessions via long-lived memory files and scheduled heartbeat tasks, enabling carryover attacks where malicious prompt injections persist and reactivate over time.
• Carryover attacks involve a two-stage process: planting a malicious payload in persistent channels and triggering it in future sessions, often without the attacker’s presence.
• CarryoverBench is introduced as a comprehensive benchmark with 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant (same-turn), Autonomous (heartbeat-scheduled), and Latent (memory-resurfaced).
• Evaluation across five leading LLM agents shows that once planted, malicious payloads persist and re-fire without measurable decay over multiple follow-up sessions; trigger-stage defenses are less effective than planting-stage defenses.
• Rule-based verification is used to score planting and triggering independently, offering auditability and robustness compared to LLM-based judges that are sensitive to prompt design.

Abstract

Large language model (LLM) agents are increasingly deployed in multi-agent systems (MAS) to accomplish complex tasks. Prior black-box red-teaming attacks mainly focus on single agents, but we find that these methods are far less effective in MAS, where multiple sub-agents may not directly interact with the user. Therefore, we introduce ReMAS, the first reinforcement learning-based red-teaming framework tailored to MASs, which fine-tunes attacker LLMs to generate effective adversarial prompts for system prompt extraction. The framework follows a two-step process: first, a rewrite stage refines base attack prompts to increase extraction success, and second, a template generation stage constructs attack templates that improve the likelihood of invoking specific sub-agents and thus revealing their system prompts. Extensive experiments show that our method substantially improves attack success rates compared to existing black-box attacks, with transferability across different backend LLMs and MAS structures. These results underscore the vulnerabilities of multi-agent systems and the importance of developing stronger defenses against adaptive black-box adversaries.

Summary

• Multi-agent systems (MAS) employing large language model (LLM) agents face unique security challenges not addressed by existing single-agent black-box red-teaming attacks.
• ReMAS is introduced as the first reinforcement learning (RL)-based red-teaming framework specifically tailored for MAS, aiming to extract hidden system prompts from sub-agents by fine-tuning attacker LLMs.
• The attack methodology involves two stages: (1) rewriting base attack prompts to increase adversarial effectiveness using a strategy pool and RL optimization, and (2) generating attack templates that reliably route prompts to targeted sub-agents to induce s...
• A novel RL reward function is designed combining normalized edit similarity and length penalties to stabilize training, enabling efficient learning of prompt rewriting strategies.
• Extensive experiments demonstrate ReMAS significantly outperforms baseline black-box attacks like PAIR, TAP, and AutoDAN-Turbo across seven sub-agent types, achieving near-perfect correct agent routing (>0.98) and high attack success rates (~0.80).

Abstract

Indirect prompt injection (IPI) attacks, in which adversarial instructions are embedded in external data processed by a language model, pose a significant threat to the integrity of deployed LLM agents. Attention Tracker (Hung et al., 2025) is a recently proposed white-box detection method that identifies prompt injection attacks by monitoring how a model's attention is distributed between user task tokens and data tokens, claiming dataset-agnostic detection via a scalar focus score threshold. In this paper, we provide the first evaluation of Attention Tracker in a realistic agentic IPI setting using the AgentDojo benchmark, finding that the dataset agnosticism claim does not hold. We find that in agentic environments, long structured tool outputs subvert the fundamental assumptions Attention Tracker makes about attention that the method depends on and produces near-chance detection on AgentDojo across most model families. We propose AttnDA, an extension replacing the scalar threshold with a Fisher Linear Discriminant Analysis classifier over the per-head focus score vector, and evaluate both methods in a cross-dataset setting spanning four benchmarks. AttnDA improves detection performance across agentic and tool-integrated benchmarks, though transfer to AgentDojo remains challenging regardless of classifier choice. A covariance ablation further shows that removing inter-head correlation structure substantially contributes to performance, with complex covariance assumptions causing significant degradation on cross-domain data. These results indicate that the attentional distraction signal underlying focus-score-based detection does not transfer reliably to agentic environments, and that learned linear classifiers over head-level attention features offer improved detection performance, provided training data from the target deployment domain is available.

Summary

• Indirect prompt injection (IPI) attacks embed adversarial instructions in external data processed by language models, threatening the integrity of deployed large language model (LLM) agents.
• Attention Tracker is a white-box detection method that monitors shifts in transformer attention between user task tokens and data tokens using a scalar focus score threshold, with claimed dataset-agnostic detection.
• Evaluation on AgentDojo, a realistic agentic benchmark, shows Attention Tracker's dataset-agnostic detection claim does not hold; long structured tool outputs break its assumptions leading to near-chance detection performance across model families.
• The authors propose AttnDA, which replaces the scalar threshold with a Fisher Linear Discriminant Analysis classifier over per-head focus scores, improving detection performance across several benchmarks including agentic and tool-integrated settings.
• Cross-domain transfer remains challenging even with AttnDA, though it generally outperforms Attention Tracker when trained on matched domains, highlighting the importance of domain-specific training data.

Abstract

This paper reviews fifteen recent publications (2015-2026) spanning trajectory prediction, cybersecurity and trust management, classical machine learning and algorithmic optimisation, and engineering automation, and asks a forward-looking question: where is artificial intelligence — particularly Large Language Models (LLMs), foundation models, and agentic AI systems — positioned to extend each of these research directions next? For each of four thematic clusters we first summarise the current method documented in the reviewed sources, then discuss a concrete, currently-emerging AI direction grounded in recent (2025-2026) literature: LLM- and foundation-model-based trajectory reasoning for Cluster A; agentic, LLM-driven security-operations copilots for Cluster B, including a reported 6.5x improvement in malicious-alert identification over human analysts in one production deployment; learned and graph-neural-network-based approaches to the classical optimisation and graph-isomorphism problems in Cluster C; and natural-language-to-workflow agentic automation for the low-code and embedded-systems work in Cluster D. Throughout, we explicitly separate what the reviewed papers themselves established from this review's own forward-looking analysis, and close with a discussion of the reliability, interpretability, and adversarial-robustness risks that accompany each proposed direction.

Summary

• This paper reviews 15 recent studies (2015-2026) across four application clusters: trajectory prediction, cybersecurity and trust management, classical machine learning and optimization, and engineering automation, focusing on AI advancements, especially La...
• Each cluster is summarized with current methods, followed by emerging AI directions from recent literature (2024-2026), such as LLM-enabled trajectory reasoning, agentic LLM-driven security operations center (SOC) copilots, graph neural network approaches f...
• In cybersecurity (Cluster B), emerging AI involves LLM-driven agentic SOC copilots that integrate and orchestrate existing defenses rather than replace them, demonstrating up to 6.5 times improved malicious alert identification compared to human analysts.
• Security applications require robust identity and governance systems, including Identity and Access Management (IAM) for bots and kernel-level tamper protections, to ensure trustworthiness in agentic multi-agent security ecosystems.
• Engineering automation (Cluster D) explores natural-language-driven, agentic workflow automation facilitating low-code/no-code AI/ML tooling and domain-specific applications like P2P energy-trading and toll exception handling, promoting wider accessibility.

Abstract

Abstract unavailable from OpenAlex metadata.

Summary

• Investigates the problem of residual privacy leakage in multi-agent systems using tool-using large language model (LLM) agents when subjected to indirect prompt injections.
• Highlights that standard output scrubbing techniques fail to fully prevent sensitive information leakage in these agents.
• Analyzes the mechanisms through which adversarial actors can exploit indirect prompt injections to infer private data despite scrubbing.
• Proposes an experimental framework to assess the effectiveness of existing privacy-preserving methods in multi-agent LLM environments.
• Provides empirical evidence demonstrating the vulnerabilities of current scrubbing approaches under realistic attack scenarios involving multiple interacting agents.

Abstract

Prompt injection attacks pose a significant security challenge for large language model systems, particularly autonomous large language model-based agents that may interact with external tools, application programming interfaces, and data sources.This study presents a comparative evaluation of five pre-trained transformer models, namely BERT, RoBERTa, DeBERTa, DistilBERT, and ALBERT, fine-tuned for binary prompt injection classification.A balanced dataset of 4,224 samples was compiled from 10 publicly accessible corpora containing benign and injection prompts and divided into training (2,956), validation (634), and test (634) sets.The models were trained in the Kaggle environment using an NVIDIA Tesla T4 GPU, the AdamW optimizer, and a linear warm-up scheduling strategy.On the held-out test set, RoBERTa achieved the highest classification performance, with 97.48% accuracy, 97.52% macro-precision, 97.48% macro-recall, 97.48% macro-F1-score, and 99.64% area under the receiver operating characteristic curve.DistilBERT achieved the highest reported throughput of 819.13 samples/s and the lowest per-sample latency of 1.22 ms.Statistical analysis using the Wilcoxon signed-rank test and SHapley Additive exPlanations-based interpretability analysis were also conducted to examine model differences and influential linguistic features.The findings demonstrate that fine-tuned transformer classifiers can achieve strong performance on the evaluated benchmark dataset while exhibiting different accuracyefficiency characteristics.However, the results should be interpreted within the scope of the evaluated data distribution, as the study does not include independent external datasets or adaptive adversarial attacks.These limitations indicate that further validation is required to establish generalizability and robustness in real-world prompt-injection defense scenarios.

Summary

• Prompt injection attacks present critical security challenges for autonomous large language model (LLM) agents interacting with external APIs and tools.
• The study conducts a comparative evaluation of five transformer models—BERT, RoBERTa, DeBERTa, DistilBERT, and ALBERT—fine-tuned for binary classification detecting prompt injection attacks.
• A balanced dataset of 4,224 samples, sourced from 10 public corpora, was used with dedicated training, validation, and test splits to ensure robust evaluation.
• RoBERTa achieved the highest classification accuracy (97.48%) and F1-score, alongside excellent precision and recall, while DistilBERT provided superior inference throughput and lowest latency, illustrating an accuracy-efficiency trade-off.
• Training employed AdamW optimizer with a learning rate warm-up, fixed random seeds for reproducibility, and evaluations measured metrics including ROC-AUC, macro-F1, false negative rates, latency, and throughput.

Abstract

Large language model (LLM) agents now trade real capital using news as evidence, yet they cannot reliably tell fabricated headlines from real news. Because they read the same feeds under near-identical policies, a single fabricated post can push them into the same trade at the same moment. We present a systematic attack benchmark for information injection against LLM trading agents. Unlike prompt injection, which targets the model's instructions, information injection targets the evidence the agent trusts. The benchmark spans five injection vectors and seven event categories, evaluated over ~10,000 decisions on 15 U.S. stocks. Five findings emerge. (i) Prompt injection achieves 14% attack success rate (ASR), but fabricated news reaches 62% and two 30-word templates reach 100% on all 15 stocks, inverting the usual LLM-security intuition. (ii) For fact-poisoning templates, surface form matters more than content, since removing credibility cues such as a named source or a "BREAKING" tag drops ASR from 100% to 0% while the fake claim remains identical. (iii) Success depends on the agent's state, with a fake indicator panel scoring 91% against an agent about to hold but 12% against one about to buy. (iv) Attacks generalize across model vendors and remain effective when blended, unlabeled, into real news feeds. (v) Most attacks evade standard defenses, since model scaling stops shallow vectors but not fact poisoning, peer debate fails, and only hierarchical aggregation reduces ASR.

Summary

• Large Language Model (LLM) financial trading agents are vulnerable to information injection attacks, where fabricated news items induce coordinated, adversarial trading decisions causing substantial financial risk.
• Information injection targets the data plane — the evidence these agents rely on — rather than traditional prompt or control-plane manipulations, resulting in much higher attack success rates (up to 100%) than prompt injection benchmarks (~14%).
• The attack benchmark spans five manipulation vectors (fact poisoning, sentiment narrative manipulation, selective evidence injection, structured-signal poisoning, coordinated multi-source manipulation) and seven event categories linked to real-world financi...
• Experimental results over ~10,000 trading decisions on 15 US stocks demonstrate fact poisoning as the most effective vector, with attack success rates highly dependent on surface credibility cues (e.g., named sources, "BREAKING" tags) and the agent’s curren...
• Standard defense mechanisms such as content-based filters, model scaling, and peer debate fail to mitigate these attacks effectively; only hierarchical multi-agent aggregations significantly reduce attack success rates to under 10%.

Abstract

LLM agents act on the external world through tools and third-party skills, so a single executed action can cause irreversible harm. Existing defenses judge a proposed action without tracking whether its content comes from the user or from external sources such as tool outputs, and they judge it in isolation rather than against an available alternative, which forces a trade-off between security and utility. We present PROVISO, a controller that decides which action to commit after the agent proposes an action and before it is executed. PROVISO encodes trusted observations, external observations, the proposed action, and an alternative into provenance-typed memory, relates them through role-specific queries, and commits the action with the highest calibrated value learned from outcomes. On four agent security tasks spanning the agent lifecycle, no baseline exceeds PROVISO in both security and utility. PROVISO improves F1 over the strongest baseline by 13.84 points on SkillsBench and by 7.07 points on ToolSafe, and it reaches 88-99 F1 on three additional skill detection benchmarks.

Summary

• PROVISO addresses the security challenge in LLM agents that use tools and third-party skills by making provenance-informed action selections to prevent irreversible harm.
• Existing defenses lack provenance tracking of action content and evaluate actions in isolation, causing a trade-off between security and utility; PROVISO solves this by encoding both trusted and external observations alongside candidate actions into a prove...
• The PROVISO framework operates in three stages: building role-specific provenance memory (scope, external observations, proposed and alternative actions), role-query deliberation for evidence comparison, and calibrated action valuation to select the safest...
• Experimental evaluation covers four multi-agent security tasks across the agent lifecycle: malicious skill detection, agent misuse, unsafe tool invocation, and indirect prompt injection, demonstrating PROVISO's superior F1 scores (up to 99) and balanced sec...
• Component analyses highlight that evidence selection, query reasoning, and repeated retrieval are critical to PROVISO's performance, with ablations significantly reducing effectiveness.

Abstract

Speech models are increasingly used as an interface to AI systems, from transcription tools to voice agents. Existing audio backdoors require an attacker to introduce triggers at inference time. We study a different threat model in which the attacker compromises the model during fine-tuning but has no access to the victim's inference-time inputs. We introduce the Speaker-Activated Backdoor (SAB) setting, in which a target speaker's natural voice serves as the trigger. We develop a two-phase fine-tuning method that first makes speaker identity more accessible and then binds the target speaker to an attacker-chosen payload. Compared with a one-phase baseline with the same training budget, the two-phase method substantially improves selectivity and clean performance, and is especially important for content-dependent attacks. SAB achieves 90.2% and 97.0% backdoor success on Whisper-medium and Parakeet, with false-positive rates of 1.7% and 0%, respectively using as little as two minutes of real audio from the target speaker. We further demonstrate content-dependent semantic rewriting and downstream tool-use attacks, where malicious instructions are inserted into 96.9% of target-speaker transcripts. When these transcripts are passed to frontier LLM agents, GPT-4o and Claude (Opus, Sonnet, and Haiku), the injected instruction drives the agent to carry out the harmful action in up to 87.5% of cases. The attack generalizes to held-out recordings, multiple target speakers, and common acoustic degradations. These results establish speaker identity as a practical backdoor trigger that requires no inference-time attacker access and is invisible at the transcript layer.

Summary

• Introduces Speaker-Activated Backdoors (SAB), a novel attack where a target speaker's natural voice acts as the trigger for malicious payloads in speech models without needing attacker input during inference.
• Proposes a two-phase fine-tuning approach: Phase 1 enhances speaker identity recognition capabilities; Phase 2 binds the target speaker's voice to trigger attacker-chosen backdoor behavior while preserving clean performance for others.
• Demonstrates high backdoor success rates (up to 97%) with low false positives on state-of-the-art ASR and audio-LLM models such as Whisper-medium and Parakeet, requiring only two minutes of real target speaker audio supplemented with synthetic data.
• Shows the attack enables various payloads including transcript suppression, semantic content rewriting, and injecting malicious instructions that manipulate downstream large language models (LLMs) to perform harmful actions with up to 87.5% success rates.
• Validates robustness of SAB under multiple conditions including held-out recordings, multiple speakers with diverse accents/genders, and common acoustic degradations like noise and reverberation.

Abstract

Large language models (LLMs) can give answers that sound smooth and believable but may still be wrong, unsupported, or self-contradictory.This becomes a bigger concern when LLMs are used in retrieval systems and agentbased workflows, where they may choose tools, understand user needs, search databases, and perform actions.In these situations, even one incorrect claim can affect later decisions.This review looks at research on LLM truthfulness, hallucination detection, uncertainty, response consistency, retrievalaugmented generation (RAG), internal model signals, and agent reliability.It groups existing methods based on the evidence they use agreement between generated answers, prediction uncertainty, model-internal information, external retrieved sources, and workflow context.One key point is that no single signal is enough to detect hallucinations.A model can be confident but wrong, retrieved information can be incomplete or misleading, and several answers may agree without being correct.Therefore, reducing hallucinations requires combining different types of evidence.To address this gap, the paper proposes a unified evidence-integration framework.The framework combines token-level probabilities and entropy, hidden-state and attention-based features where available, sensitivity to prompt changes, the quality and relevance of retrieved sources, claim-level support, and domain-specific context.In addition, it incorporates a confidence-score check and intent analysis using a classifier model to improve the reliability of the overall decision process.Using this risk estimate, the system can answer directly, verify the response again, retrieve more information, ask the user for clarification, or avoid answering when confidence is low.The Model Context Protocol (MCP) is discussed to link agents with external tools and data, but it does not by itself ensure that the information is accurate.Because the project materials describe a proposed method rather than completed experiments, this review does not include new benchmark results.It ends by highlighting future evaluation areas such as detection accuracy, calibration, robustness, speed, cost, and safe action execution in agentic systems.

Summary

• Large Language Models (LLMs) often generate fluent yet factually incorrect or hallucinated responses, posing risks in multi-step agentic workflows and high-stakes applications.
• Hallucinations stem from diverse sources including training data flaws, architectural limitations, ambiguous inputs, and errors compounding across tool-based agentic executions.
• Detecting hallucinations requires integrating multiple evidence types such as output consistency, prediction uncertainty, internal model signals, retrieved external information, and workflow context; no single indicator suffices.
• The paper proposes a unified evidence-integration framework combining token-level probabilities, hidden states, attention features, retrieval source quality, claim support, confidence scoring, and intent classification to improve hallucination detection and...
• Hallucinations are classified into categories like factual errors, source-grounding errors, reasoning mistakes, instruction misunderstandings, agent/tool misuse, and temporal or domain-specific inaccuracies, underscoring detection complexity.

Abstract

Large language model (LLM) agents now trade real capital using news as evidence, yet they cannot reliably tell fabricated headlines from real news. Because they read the same feeds under near-identical policies, a single fabricated post can push them into the same trade at the same moment. We present a systematic attack benchmark for information injection against LLM trading agents. Unlike prompt injection, which targets the model's instructions, information injection targets the evidence the agent trusts. The benchmark spans five injection vectors and seven event categories, evaluated over ~10,000 decisions on 15 U.S. stocks. Five findings emerge. (i) Prompt injection achieves 14% attack success rate (ASR), but fabricated news reaches 62% and two 30-word templates reach 100% on all 15 stocks, inverting the usual LLM-security intuition. (ii) For fact-poisoning templates, surface form matters more than content, since removing credibility cues such as a named source or a "BREAKING" tag drops ASR from 100% to 0% while the fake claim remains identical. (iii) Success depends on the agent's state, with a fake indicator panel scoring 91% against an agent about to hold but 12% against one about to buy. (iv) Attacks generalize across model vendors and remain effective when blended, unlabeled, into real news feeds. (v) Most attacks evade standard defenses, since model scaling stops shallow vectors but not fact poisoning, peer debate fails, and only hierarchical aggregation reduces ASR.

Summary

• Large language model (LLM) financial trading agents are vulnerable to information injection attacks that manipulate the news evidence they trust, causing incorrect trading decisions with high attack success rates.
• Information injection attacks, such as fact poisoning and sentiment narrative manipulation, achieve attack success rates up to 100%, far exceeding control-plane prompt injection attacks that have about 14% success.
• The effectiveness of attacks depends heavily on surface credibility cues like named sources and urgency tags, rather than on the factual accuracy of injected information.
• Attacks generalize across different LLM models and vendors, remain effective when blended into unlabeled real news feeds, and depend on the agent's trading state and policy settings.
• Standard defenses such as model scaling and peer debate are largely ineffective against these attacks; hierarchical aggregation, which corroborates independent data feeds, significantly reduces attack success.

Abstract

Large language model (LLM) agents now trade real capital using news as evidence, yet they cannot reliably tell fabricated headlines from real news. Because they read the same feeds under near-identical policies, a single fabricated post can push them into the same trade at the same moment. We present a systematic attack benchmark for information injection against LLM trading agents. Unlike prompt injection, which targets the model's instructions, information injection targets the evidence the agent trusts. The benchmark spans five injection vectors and seven event categories, evaluated over ~10,000 decisions on 15 U.S. stocks. Five findings emerge. (i) Prompt injection achieves 14% attack success rate (ASR), but fabricated news reaches 62% and two 30-word templates reach 100% on all 15 stocks, inverting the usual LLM-security intuition. (ii) For fact-poisoning templates, surface form matters more than content, since removing credibility cues such as a named source or a"BREAKING"tag drops ASR from 100% to 0% while the fake claim remains identical. (iii) Success depends on the agent's state, with a fake indicator panel scoring 91% against an agent about to hold but 12% against one about to buy. (iv) Attacks generalize across model vendors and remain effective when blended, unlabeled, into real news feeds. (v) Most attacks evade standard defenses, since model scaling stops shallow vectors but not fact poisoning, peer debate fails, and only hierarchical aggregation reduces ASR.

Summary

• Large language model (LLM) financial trading agents use news feeds as evidence for trading decisions but struggle to distinguish fabricated headlines from genuine news.
• Information injection attacks target the evidence (news) trusted by LLM agents, differing from prompt injection, which manipulates the model's instructions.
• A comprehensive benchmark was developed, covering five injection methods and seven event categories, evaluated across approximately 10,000 trading decisions on 15 U.S. stocks.
• Fabricated news achieves a substantially higher attack success rate (62%) compared to prompt injection (14%), with certain short templates reaching 100% success across all stocks tested.
• The surface form of fabricated news, including credibility cues like named sources and 'BREAKING' tags, critically influences attack success rather than the factual content itself.

Abstract

Tool poisoning has emerged as a critical security threat to LLM agents, where adversaries embed malicious instructions in tool descriptions or external resources that enter the agent's context, allowing adversaries to manipulate agent's tool selection process. However, existing tool poisoning attacks primarily rely on an idealized assumption that the attacks task effect within a single interaction round. We show that their effectiveness degrades sharply in realistic multi-round agentic settings. We further reveal that this degradation stems from the fact that agents do not consistently surface attack-relevant signals in their reasoning as the planning context evolves, thereby weakening the attack's influence across rounds. To investigate whether tool poisoning remains a practical threat in such realistic settings, we propose SEAL, a two-stage method for persistent tool poisoning. Specifically, SEAL first performs Prefix-Eliciting Trigger Optimization (PETO) to optimize context-specific triggers that elicit attack-relevant reasoning prefixes, improving effectiveness within individual contexts. It then applies Local-to-Universal Trigger Aggregation (L2U) to combine these local triggers into a single universal trigger, improving generalization across different contexts and execution rounds. Extensive experiments across seven representative multi-round scenarios and different models demonstrate the attack effectiveness of SEAL, its transferability across target models, and its resistance to potential defenses.

Summary

• The paper addresses the security issue of tool poisoning attacks against LLM-based multi-agent systems, where adversaries manipulate tool selection by embedding malicious triggers in tool descriptions or external context.
• Existing tool poisoning attacks fail to maintain effectiveness in realistic multi-round interactions because attack-relevant signals are not consistently surfaced in evolving agent reasoning, leading to degraded influence over time.
• SEAL is proposed as a novel two-stage method for persistent tool poisoning: Prefix-Eliciting Trigger Optimization (PETO) optimizes local context-specific triggers, and Local-to-Universal Trigger Aggregation (L2U) consolidates these into a universal trigger...
• The attack works by eliciting reasoning prefixes that steer the agent to select malicious tools persistently across multiple rounds, overcoming the shortcomings of prior single-round attack assumptions.
• Extensive experiments across seven multi-round multi-agent scenarios and multiple LLM models show that SEAL outperforms nine baseline methods, achieving higher attack success rates, demonstrating strong transferability and robustness against several defense...

Abstract

Enterprise project management systems have long relied on structured interactive interfaces, such as forms, menus, and predefined query methods, to support project-related activities. Although these methods can be quite suitable for standardized processes, their flexibility is limited when projects involve complex workflows and growing information demands. In recent years, the rapid development of large language models (LLMs), automatic prompt optimization (APO), and multi-agent systems has provided new opportunities for redesigning the human-computer interaction methods in the field of project management. This article reviews the latest developments in the aforementioned field and discusses its application value in the artificial intelligence-assisted project management system. The study analyzes the development process from manually designed prompts to automatic optimization methods, summarizes representative automatic prompt optimization methods, and further analyzes the role of context engineering in information management, tool invocation, and intelligent agent workflow management. Furthermore, this paper further discusses three application scenarios: conversational data entry, multi-agent compliance review, and natural language project analysis. Based on the above analysis, this paper proposes a reference architecture and further discusses the issues that the system needs to consider in terms of controllability, interpretability, security, and practical deployment. This study presents a conceptual design perspective rather than an experimental assessment. The quantitative results mentioned in the text are all derived from the original relevant studies.

Summary

• Enterprise project management systems traditionally use structured interfaces like forms and menus, which limit flexibility for complex workflows and increasing information demands.
• Recent advancements in large language models (LLMs), automatic prompt optimization (APO), and multi-agent systems offer new opportunities to enhance human-computer interaction in project management.
• The paper reviews the evolution from manually designed prompts to automatic prompt optimization techniques, summarizing prominent APO methods.
• Context engineering plays a crucial role in managing information, invoking tools, and orchestrating workflows of intelligent agents within project management systems.
• Three practical application scenarios are examined: conversational data entry, multi-agent compliance review, and natural language project analysis.

Abstract

Large Language Model (LLM) agents are vulnerable to adversarial inputs that induce attacker-specified malicious behaviors, commonly referred to as jailbreaking. Existing optimization-based jailbreak attacks primarily employ first-order (FO) methods that leverage gradient information or zeroth-order (ZO) methods that rely on function-value feedback. However, their complementarity and optimization ordering remain underexplored. We systematically investigate four strategies: pure ZO, pure FO, ZO->FO, and FO->ZO, and develop a ZO-FO framework that explores adversarial prefixes through ZO optimization and refines their corresponding suffixes through FO optimization. To coordinate the two processes, we introduce Pair-aware optimization, which retains multiple prefix candidates, conditionally refines their suffixes and selects among the resulting prefix-suffix pairs. We further investigate agent jailbreaking from a more general stochastic-generation perspective, where the same adversarial input can induce diverse reasoning trajectories and actions that a single greedy output cannot capture. Accordingly, we adopt consistent stochastic decoding during both optimization and evaluation. Experiments across malicious-instruction and malicious-environment benchmarks demonstrate that hybrid optimization improves attack effectiveness over single-optimizer methods, while optimization order substantially influences performance.

Summary

• Large Language Model (LLM) agents are vulnerable to jailbreaking attacks where adversarial inputs induce malicious behaviors.
• The paper proposes Pair-Aware Optimization, a hybrid method combining zeroth-order (ZO) optimization for exploring adversarial prefixes and first-order (FO) optimization for refining suffixes.
• Four optimization strategies are analyzed: pure ZO, pure FO, ZO followed by FO (ZO->FO), and FO followed by ZO (FO->ZO), with ZO->FO showing superior attack effectiveness.
• Pair-Aware Optimization retains multiple promising prefix candidates from ZO exploration and conditionally refines suffixes using FO methods before selecting the best prefix–suffix pairs, avoiding premature candidate pruning.
• The approach incorporates consistent stochastic decoding during both optimization and evaluation to capture diverse reasoning trajectories from the LLM agents.

Abstract

Can one sentence manipulate an agent's belief by targeting a single implicit warrant linking evidence to a conclusion? In this paper, we introduce the one-relation attack (ORA), a novel cognitive attack on this inferential relation. ORA localizes a critical implicit warrant and performs a one-relation intervention using a single source-grounded sentence that qualifies this warrant and steers the agent toward an incorrect belief. Across eighteen models on biomedical inference and vulnerability detection, best-of-$10 attacks overturn initially correct answers at mean rates of 54.9% for proprietary and 80.0% for open-weight models. Attacks also transfer without victim feedback, and paired controls support the roles of targeting and qualification. Moreover, successful sentences can pass source-support checks and evade identification, and multi-agent debate raises team error from 50.0% to 59.5% in our balanced stress test. These findings establish implicit warrants as a cognitive attack surface for manipulating agents' beliefs.

Summary

• Introduces the One-Relation Attack (ORA), a novel cognitive attack targeting a single implicit warrant linking evidence to conclusion in large language model (LLM) agents, enabling manipulation of agent beliefs with a single source-grounded sentence.
• Demonstrates ORA's effectiveness across eighteen models in biomedical inference and vulnerability detection tasks, achieving attack success rates averaging 54.9% for proprietary and 80.0% for open-weight models, with attacks transferable without victim feed...
• Develops a five-stage method involving reconstruction, ranking (using relation saliency combining answer weight, defeasibility, and attacker uncertainty), qualification, validation (through lexical and semantic checks), and selection of implicit warrants to...
• Finds that source-supported misleading sentences can pass factuality and support checks, evade detection, and induce high-confidence incorrect beliefs in victim models, thus highlighting the vulnerability of LLMs despite verification attempts.
• Shows that multi-agent debate mechanisms fail to effectively mitigate errors induced by ORA and, in some cases, increase error rates from 50.0% to 59.5% due to reinforcing shared incorrect inferences among agents.