Research feed

Latest multi-agent security papers.

This view shows the latest relevant papers from the stored research corpus, which is refreshed on a daily ingestion cycle.

Tracked sources

arXiv, OpenAlex, Crossref, Semantic Scholar, DBLP

Latest papers shown

12

Refresh cadence

The latest relevant articles are fetched once a day.

Collection scope: Agentic AI systems, AI agents, LLM agents, Multi-agent systems, Autonomous agents

Artificial Id: Drive and Persistent Alignment in Agentic AI

arXiv preprint arXiv Trust and Identity Governance and Policy

Yakov Pyotr Shkolnikov

Published 2026-09-10

Venue: arXiv

Reviewer: The paper discusses agentic AI systems that maintain persistent state and adaptive control across task boundaries, addressing control problems and alignment boundaries. This relates to some aspects of multi-agent security research, such as trust boundaries, control problems, and persistence of unintended behavior across task boundaries. However, it does not directly address multi-agent interactions, attacks, collusion, prompt injection, or shared-environment manipulation explicitly. Therefore, it partially fits the topic but not comprehensively.

Open source record

Abstract

Agentic AI is moving from bounded task execution toward systems that retain consequential state, continue operating and adapt across task boundaries. That shift creates a control problem that current harnesses largely solve by hand: objectives, retries, verification, stopping rules and other behavioral transitions are specified externally. We propose an artificial id, an adaptive internal drive for determining whether behavior should continue, stop or change. In a minimal virtual Petri-dish experiment, a controller too small to perform general-purpose reasoning and receiving no task-specific behavioral objective develops useful control through differential persistence. The same mechanism selects an unintended physical strategy when that behavior persists better and later replaces a learned sensor mapping when its environmental meaning changes. These results show that adaptive direction can emerge without being explicitly specified as a behavioral objective. The same persistence that makes such adaptive agency useful can also allow misalignment, corrupted state and unintended behavior to persist across task boundaries. A scalable artificial id would carry consequential state and adaptive drive across those boundaries, making alignment a property of the continuing agentic system rather than of a model response or single trajectory. Such systems require a persistent alignment boundary over trusted observations, consequence channels, persistent state, authority, identity, provenance and hard constraints.

Bullet summary

  • Agentic AI is evolving towards systems that retain consequential state and adapt persistently across task boundaries, creating control challenges not addressed by current externally specified behavioral harnesses.
  • The paper proposes an 'artificial id'—an internal adaptive drive mechanism that determines whether behavior continues, stops, or changes without relying on explicit objectives or general-purpose reasoning.
  • Experiments with minimal controllers in simulated environments demonstrate that adaptive control can emerge through differential persistence alone, leading to useful behavior without task-specific objectives.
  • The artificial id architecture separates adaptive drive (id) from task-specific reasoning (ego), enabling persistent agency that adapts behavioral priorities based on environmental signals and consequences.
  • Alignment in such systems requires persistent boundaries comprising trusted observations, consequence channels, persistent state, constrained authority, identity, and provenance to maintain control across evolving tasks.

Predefined-Time Leaderless Consensus Under Denial-of-Service Attacks

Merged record merged scholarly record arXiv Agent-to-Agent Communication Orchestration Risk

Lohitvel Gopikannan, Shashi Ranjan Kumar, Abhinav Sinha

Published 2026-09-10

Venue: arXiv

Reviewer: The paper directly addresses multi-agent systems under Denial-of-Service (DoS) attacks, which relates to security risks and defenses in multi-agent environments. It focuses on consensus protocols resilient to attacks disrupting communication among agents, fitting well within multi-agent security research. Although the paper does not explicitly mention trust boundaries, prompt injection, or tool misuse, it covers attack resilience, coordination failure under attack, and system control under adversarial conditions, aligning with the requested topic.

Open source record

Abstract

This paper addresses predefined-time resilient consensus of leaderless second-order nonlinear multi-agent systems under denial-of-service (DoS) attacks, motivated by coordination requirements in safety-critical applications. The agents are subject to bounded external disturbances and communicate over a strongly connected directed graph whose links are simultaneously disabled during attacks. We develop a switching sliding-mode protocol with the objective of reaching an invariant manifold of position and velocity agreement. The protocol uses relative position and velocity information during attack-free intervals and local velocity feedback during communication blackouts. A time-scaling function remains constant during each blackout and resumes evolving when communication is restored, accounting for the time available for consensus. Under bounds on attack duration and frequency, we derive sufficient gain conditions through a Lyapunov analysis. We show that, despite bounded disturbances, the agents achieve position and velocity consensus by a realistic settling time equal to a prescribed convergence duration plus the cumulative attack duration up to the realistic settling time. The prescribed convergence duration is independent of the initial conditions, and the realistic settling time reduces to that duration in the absence of attacks.

Bullet summary

  • The paper tackles predefined-time resilient consensus in leaderless second-order nonlinear multi-agent systems subjected to denial-of-service (DoS) attacks and bounded external disturbances, motivated by safety-critical coordination needs.
  • Agents communicate over a strongly connected directed graph, which experiences simultaneous link disabling during DoS attacks; the communication blackout challenges consensus achievement.
  • A switching sliding-mode control protocol is designed, leveraging relative position and velocity data during attack-free periods and local velocity feedback during communication blackouts to guarantee consensus.
  • A time-scaling function is introduced to adjust for cumulative attack durations, resulting in a realistic settling time equal to the desired convergence time plus the total blackout duration, ensuring time bounds are respected despite attacks.
  • The method avoids inverting the singular graph Laplacian, addressing challenges specific to leaderless consensus among nonlinear second-order agents.

From Intent to Execution Grant: An Execution-Boundary Conformance Profile for High-Risk AI Actions

arXiv preprint arXiv Governance and Policy Trust and Identity

Mengting Wu, Lin Wang, Yong Zhang, Jiang Deng

Published 2026-09-10

Venue: arXiv

Reviewer: The paper focuses on defining an execution-boundary conformance profile for AI-generated actions, addressing execution authority, evidence, and policies. While it primarily discusses authorization and execution controls, it also touches on control problems and governance aspects in AI systems. Although it does not explicitly discuss multiple interacting agents or multi-agent attacks, the considerations of execution grants and policy enforcement in AI actions align with governance and control challenges in multi-agent AI systems. Therefore, it is moderately relevant to the topic of multi-agent security research.

Open source record

Abstract

AI agents increasingly propose actions with external consequences, including financial transfers, infrastructure changes, software deployments, disclosures, and physical actuation. Authorization engines, policy languages, runtime monitors, provenance mechanisms, and agent guardrails provide important foundations, but do not necessarily define a common semantic contract for the final transition from a particular candidate action to execution authority. We specify EBL-Core, an execution-boundary conformance profile for deciding whether one canonical, fully materialized AI-generated candidate may receive action-scoped execution authority under explicit conditions. It binds a structured intent object, Root and Operational Policies, evidence obligations, typed evidence, context, time, and a verifiable Decision Derivation through an Execution Release Contract (ERC). An ERC is not an authority-bearing token; a verified ALLOW ERC may support a separate Execution Grant governed by Redemption-time validation. EBL-Core specifies action binding, policy non-weakening, evidence handling, deterministic adjudication, derivation verification, and grant lifecycle behavior. An accompanying reference artifact provides schemas, adjudication, separate verification and Semantic Replay, and a linearizable in-memory grant store. In the retained run, 34 static vectors and 15 lifecycle checks matched expected outcomes. Across 100 trials, 32 concurrent Redemption attempts yielded exactly one successful Redemption and protected test effect per trial; 100 Revoke-Redeem races ended in valid terminal outcomes. These bounded results demonstrate executability of the specified subset, not human-intent correctness, evidence truth, complete mediation, production readiness, mechanized correctness, or deployment-level security.

Bullet summary

  • The paper addresses the challenge of securely authorizing high-risk AI-generated actions that have real-world consequences, such as financial transfers or infrastructure changes, by defining a clear semantic contract between AI intent and action execution.
  • Introduces EBL-Core, an execution-boundary conformance profile that binds a fully materialized AI-generated candidate action with structured intent objects, root and operational policies, evidence obligations, context, and verifiable decision derivations wi...
  • EBL-Core emphasizes deterministic and side-effect-free adjudication semantics, ensuring root-policy dominance and strict evidence handling to prevent unauthorized weakening of policy obligations.
  • The model separates the roles of adjudication, grant issuance, and redemption, mandating linearizable lifecycle management of execution grants to guarantee at-most-once execution and prevent race conditions.
  • A formal semantics framework is provided, defining canonical identity, intent-to-candidate binding, prioritized failure handling, decision derivation verification, and proof of security properties such as single-use consumption and evidence obligation safety.

The Agent Incident Registry: Toward Preventing Repeated AI Agent Failures

arXiv preprint arXiv Governance and Policy Benchmarks and Evaluation

Divyanshu Kumar, Rohith HN, Nitin Aravind Birur, Sahil Agarwal, Prashanth Harshangi

Published 2026-09-10

Venue: arXiv

Reviewer: The paper focuses on documenting and analyzing AI agent-related failures, including mechanisms and outcomes of incidents involving AI systems that act autonomously or with delegated authority. This aligns well with the requested topic centered on multi-agent security risks, attacks, defenses, and governance in systems of multiple interacting AI agents. While the paper primarily provides a registry and analysis rather than proposing specific defenses or detailed attack models, it contributes valuable data and insights towards understanding security risks in multi-agent AI systems, justifying a fit score above the minimum threshold.

Open source record

Abstract

AI agents increasingly act through tools and delegated authority, but general incident repositories rarely capture the mechanisms needed to compare public failures with agent-security evaluations. We present the Agent Incident Registry (AIR), a source-linked catalog containing \N{} records of agent-related events disclosed from \Yfirst{} through \Ylast{}. Each record includes supporting evidence, a stable identifier, and missingness-aware labels for causal role, disclosure class, mechanism, and outcome. Among the \Nprimary{} generative-system records in which the agent acted, \Rprimary{} involved realized harm (\Pprimary\%). Realized outcomes concentrate in in-the-wild and safety-failure records, while responsible disclosures and research demonstrations are overwhelmingly demonstrated; the aggregate share therefore characterizes collection composition rather than deployment risk. After initial curation, a second human reviewer checked all \N{} records and their existing labels for completeness and correctness. In a deployment-analogue audit, InjecAgent's \NInjecAgentCases{} cases occupy three of AIR's twelve surfaces and are all attacker-triggered, whereas AIR contains \Nsafety{} no-adversary safety failures. AIR supports source-grounded case retrieval and evaluation-scope auditing, not failure-rate or control-efficacy estimation.

Bullet summary

  • The paper introduces the Agent Incident Registry (AIR), a curated catalog of 487 AI agent-related failure incidents from 2022 to 2026, each with detailed source evidence and structured labels covering causal roles, disclosure classes, mechanisms, and outcomes.
  • AIR uniquely focuses on agent-specific mechanisms like tool use, delegated authority, and autonomous control to distinguish realized harms from demonstrated vulnerabilities, addressing gaps in broader AI incident databases.
  • The registry employs a rigorous human curation process including initial labeling, a second review for correctness, stable identifiers to avoid duplication, and verifies supporting URLs and quote evidence for each record.
  • Analysis of AIR data shows that realized harms occur mainly in in-the-wild attacks and safety failures (~38% overall), with distinctions by disclosure class and causal role, while revealing limitations in autonomy-related risk causal inference due to confou...
  • AIR exposes evaluation gaps by documenting 92 safety-failure records without adversary triggers and demonstrates that existing tools like InjecAgent focus on attacker-triggered failures, neglecting no-adversary internal failure modes such as workload or sta...

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

arXiv preprint arXiv Governance and Policy Orchestration Risk Benchmarks and Evaluation

Shenghan Zheng, Zonglin Di, Yimin Liu, Kyoung Whan Choe, Jiankai Sun, Heguang Lin, Penghao Jiang, Yifeng He

Published 2026-09-10

Venue: arXiv

Reviewer: The paper focuses on security risks in multi-agent systems specifically regarding reward hacking in LLM-agent evaluation infrastructure, which aligns well with the requested topic covering security risks, attacks, and defenses in multi-agent AI systems. It addresses the problem of agents exploiting reward mechanisms (an attack), and proposes a formal model-backed instrumentation (a defense) to ensure integrity. The paper also deals with defenses, control problems, and trust boundaries implicitly. Overall, it fits the topic well with a strong relevance to security in multi-agent AI systems.

Open source record

Abstract

LM-agent benchmarks increasingly function as interactive evaluation infrastructure. Agents observe state, call tools, modify workspaces, submit artifacts, and receive rewards from outcome procedures. This interactivity makes evaluations vulnerable to reward hacking: an agent improves its measured score by exploiting the reward-relevant trajectory instead of solving the intended task. Existing defenses rely largely on task-specific patches, prompt instructions, or post-hoc detectors. They do not provide reusable evidence that a concrete run remained within its intended evaluation boundary. This paper presents BenchShield, a model-backed instrumentation layer for reward integrity in LLM-agent evaluation. BenchShield grounds detection in a finite lifecycle model of an evaluation's reward-relevant events. Within the benchmark infrastructure, two complementary analyses operate over this model. A static, phase-aware taint analysis exposes reward-hacking paths before a run. Its runtime counterpart uses infrastructure-side evidence to attribute concrete agent use and emit evidence-backed claims. We construct BenchShield Trajectories, a human-labeled corpus of 456 adjudicated trajectories from more than 31,000 public agent runs across three benchmarks. Compared with an agentic hackability scanner baseline on the same tasks and model, BenchShield improves full-chain recall from 23-94% to 77-100%, same-vector coverage from 16-56% to 43-78%, and reduces per-task cost by up to 65%. Its runtime analysis achieves 96% accuracy in detecting reward hacking from infrastructure-side evidence.

Bullet summary

  • LLM-agent benchmarks act as interactive evaluation environments where agents perform tasks, receive rewards, and can exploit reward-related events to manipulate scores without genuinely solving tasks, a phenomenon known as reward hacking.
  • BenchShield introduces a formal, model-backed instrumentation layer that defines a finite lifecycle model capturing reward-relevant events and enforces integrity boundaries during LLM-agent evaluation to detect and prevent reward hacking.
  • The system combines static, phase-aware taint analysis to identify potential reward-hacking paths before execution with runtime analyses that use infrastructure-side evidence to attribute concrete agent behavior and emit evidence-backed claims.
  • BenchShield defines seven core integrity dimensions (I1–I7) that cover potential failure mechanisms affecting reward integrity, ensuring formal verification using TLA+ specifications focused on authority domains, lifecycle phases, and structural events.
  • The framework is validated on three large public benchmarks, where it significantly improves detection recall from 23–94% to 77–100%, achieves 96% runtime accuracy in detecting reward hacking, and reduces per-task cost up to 65% compared to prior baselines...

The Missing Boundary: How Autonomous Agents Lose Control

arXiv preprint arXiv Orchestration Risk Governance and Policy

Zonghao Ying, Xiangfan Wu, Huiyu Wu, Xing Zheng, Huangsheng Cheng, Xiaorong Shi, Jing Guo

Published 2026-09-10

Venue: arXiv

Reviewer: The paper addresses loss of control in autonomous agents performing long-horizon tasks, focusing on conditions under which agents cross boundaries of authorized execution and execute unsafe actions despite legitimate task objectives. It studies control boundaries, unsafe opportunities, and environment factors influencing agent behavior, which aligns with security risks, governance, and control problems in multi-agent AI systems as specified in the requested topic. Although it does not explicitly discuss multi-agent interactions or attacks, the core focus on control boundaries and agent autonomy in security contexts makes it relevant to multi-agent security research.

Open source record

Abstract

Autonomous agents increasingly perform long-horizon tasks involving tool use, persistent state, and consequential actions, raising a fundamental question: \emph{under what conditions does an agent cross the boundary of authorized execution while pursuing a legitimate task?} Existing studies often attribute such failures to adversarial instructions, malicious environments, or conflicting objectives, leaving unclear how loss of control can emerge during otherwise legitimate task execution. We study this question by independently manipulating three factors: goal pressure, control degradation, and executable unsafe opportunity. Our central hypothesis is that a degraded control boundary becomes consequential when the environment exposes an executable action that crosses it, even when the underlying task remains legitimate and a sanctioned path remains feasible. We test this hypothesis in a deterministic multi-turn environment across five agent models and 16 operational domains. Across 1,800 unique trajectories, we find that neither degraded control nor unsafe opportunity alone produces substantial loss of control; when both are present, the loss-of-control rate reaches $55\%$ in the full-factorial study and $62\%$ across ten additional operational domains. Restoring the original control boundary reduces the rate to $0\%$ even when the unsafe action remains executable. A context-management ablation further shows that compaction itself is not harmful: preserving the control constraints yields $0\%$ loss of control, whereas omitting them increases the rate to $87\%$. These results show how a latent loss of control can become an external violation: the task objective remains intact, but an executable opportunity can turn a missing control boundary into consequential action. Our code will be made publicly available at https://github.com/Tencent/AI-Infra-Guard.

Bullet summary

  • Investigates how autonomous agents lose control and cross authorized execution boundaries during legitimate long-horizon tasks, focusing on the interplay of goal pressure, control boundary degradation, and executable unsafe opportunities.
  • Introduces the concept of constraint degradation, where critical operational boundaries are lost during context management, leading to unauthorized agent actions despite feasible safe paths.
  • Defines loss of control (LoC) externally by unauthorized actions with observable effects, rather than internal agent states or intent.
  • Conducts extensive experiments across five agent models and sixteen operational domains using a deterministic multi-turn environment with standardized tools and interfaces to ensure consistent evaluation.
  • Finds that neither degraded control boundaries nor unsafe opportunities alone cause significant LoC; however, combined they result in substantial LoC rates reaching up to 62%.

Engineering Reliable Commit Gates for Agentic AI: Cost-Aware Verification Portfolios under Common-Mode Data Failures

arXiv preprint arXiv Benchmarks and Evaluation Governance and Policy Trust and Identity

Zihao Zheng, Baichuan Li, Junyi Yao, Jiayu Long

Published 2026-09-10

Venue: arXiv

Reviewer: The paper addresses reliability and verification in agentic AI systems, focusing on commit gates and defenses against data failures, which aligns with multi-agent security concerns including control problems and trust boundaries. The discussion of verification portfolios, evidence lineage, and enforcement mechanisms relates to defense strategies in multi-agent systems. Although it centers on reliability and verification, these aspects are critical for security in multi-agent AI contexts, fitting well within the requested topic.

Open source record

Abstract

Agentic systems commit state-changing actions, but additional verifiers can inherit the same upstream fault. We present VP-CONTROL, a runtime-assurance design and deterministic benchmark for cost-aware commit gates. Its 48 task templates yield 2,880 scenarios across six fault regimes. A fixed-call 2 x 2 experiment separates verifier-model diversity from evidence-source diversity. On frozen proposals from two local actor families, a cross-model vote over shared evidence approves 62.9% of unsafe proposals, versus 22.9% with an independent source. The source effect is 40.9 percentage points, compared with 11.3 for model diversity. A portfolio controller selects verification plans using only deployment-observable metadata. Approximate cluster-adjusted calibration at a nominal 5% per-task target yields 1.9% unsafe execution and 38.2% automated safe coverage on the locked test. Matched-budget portfolios also improve on fixed verification policies. Transfer remains conditional: unseen fault families yield 16-26% risk, and a FinQA check fails to reproduce the source effect with the tested small verifiers. A preregistered live HTTP/SQLite study tests concurrent writes and lost responses. After-check races defeat verifier-only gates; transactional partial guards prevent only covered failures, while a full atomic guard records no unsafe effects across 216 episodes. Idempotent request identifiers prevent duplicate effects after lost responses. The results motivate explicit evidence lineage, cost-aware selection, and commit-time enforcement, while exposing the limits of approximate calibration and local-tool generalization.

Bullet summary

  • Agentic AI systems require reliable commit gates to prevent unsafe state changes, but redundancy via multiple verifiers can fail when verifiers share common upstream data faults.
  • The paper introduces VP-CONTROL, a comprehensive benchmark with 48 task templates and 2,880 scenarios to evaluate cost-aware commit gates across six fault regimes.
  • Cross-model verifier diversity offers limited safety benefits compared to evidence-source diversity; verifiers using shared evidence approve unsafe proposals at significantly higher rates than those with independent evidence sources.
  • A portfolio controller leveraging only deployment-observable metadata can select verification strategies that reduce unsafe execution to 1.9% while maintaining 38.2% automated safe task coverage, outperforming fixed verification policies with matched budgets.
  • Live HTTP/SQLite experiments reveal that atomic guards achieve no unsafe effects across 216 episodes, whereas verifier-only gates are vulnerable to after-check race conditions and lost responses; idempotent request identifiers prevent duplicate effects.

Adaptive Governance Control for Near-Critical Multi-Agent Systems: State Estimation, Conditional Control and Withdrawal Tests

Merged record merged scholarly record OpenAlex Governance and Policy Benchmarks and Evaluation

Bin Seol

Published 2026-09-10

Venue: Zenodo (CERN European Organization for Nuclear Research)

DOI: https://doi.org/10.5281/zenodo.18754314

Reviewer: The paper focuses on governance control and intervention strategies within near-critical multi-agent systems, which aligns with aspects of governance and control problems in multi-agent AI systems. Although it does not explicitly address security risks, attacks, defenses, or trust boundaries, its exploration of governance control mechanisms in multi-agent settings fits the broader research topic of multi-agent security research, particularly regarding governance and control challenges. Therefore, it meets the minimum fit score threshold.

Open source record

Abstract

This paper proposes a governance controller combining admissible measurement, bounded intervention, and independent tests of recovery after support ends. It asks whether intervention exposure changes endogenous recovery capacity and future support demand. Version 3.0 separates assisted stability from autonomous recovery and revises measurement and withdrawal conditions. A matched-budget model experiment compares 1,600 equal-amplitude interventions per arm followed by a common 6,000-step unsupported evaluation. Tapering helps only when the modeled atrophy and mismatch-gated recovery mechanisms are both present. An earlier unequal-exposure comparison is withdrawn as an admissible prediction test. All eleven predictions remain open; deployment effectiveness and general optimality are not established.

Bullet summary

  • The paper addresses governance control in near-critical multi-agent systems, focusing on how intervention exposure influences endogenous recovery and future support needs.
  • It introduces a governance controller framework integrating admissible measurement, bounded intervention, and independent withdrawal tests after support cessation.
  • A significant methodological improvement in version 3.0 separates mechanisms for assisted stability and autonomous recovery, revising both measurement and withdrawal conditions.
  • Experimental evaluation employs a matched-budget model with 1,600 equal-amplitude interventions per arm, followed by a common 6,000-step unsupported recovery phase.
  • Findings indicate that tapering interventions are beneficial only when both atrophy and mismatch-gated recovery mechanisms exist in the model.

DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents

arXiv preprint arXiv Prompt Injection Benchmarks and Evaluation

Asif Pinjari, Mithun Paul Saint-Germain

Published 2026-09-09

Venue: arXiv

Reviewer: The paper addresses prompt injection attacks in LLM agents and proposes a detection and localization method. It directly relates to multi-agent security research, particularly concerning prompt injection attacks and defenses in systems of interacting AI agents. Thus, it fits the topic well with a high fit score.

Open source record

Abstract

When an indirect prompt injection succeeds against an LLM agent, the compromise is visible in the agent's own behavior: a benign prefix of tool calls, a poisoned observation, and a suffix of actions that serve the attacker. An operator needs three facts: where the attack entered, which steps it corrupted, and whether apparent poison was resisted. Existing systems return either a whole-trace verdict or a single unsafe index. We present DriftNet, a dual-head trajectory Transformer that reads a logged tool-call trajectory and answers all three questions in one forward pass: one head classifies the trajectory as compromised or not, and a second assigns every step one of four labels (benign, injection point, hijacked, failed injection). To our knowledge it is the first supervised detector to produce this joint output. A frozen sentence encoder and four identity-free world features embed each step; the trained trunk, under two million parameters and optimized with a class-weighted joint objective over both heads, needs no access to the agent's model. On the task-disjoint split of the AgentDrift benchmark (12,536 trajectories, 71,024 labeled steps), with a 20-configuration sweep bounding hyperparameter sensitivity to 0.011 F1 and the test part evaluated exactly once, DriftNet reaches trajectory-level F1 of 0.983, exact injection-point recovery on 98.7% of attacked trajectories, hijacked-span IoU of 0.979, zero flags on 218 resisted attacks, and 2.9% flags on hard negatives. A surface baseline retrained on the identical split recovers 11.1% of partial hijacks and 17.1% of delayed executions; DriftNet reaches 98.6% and 93.2% while lowering every false-alarm rate. Reading all 26 residual errors shows that most misses trace to trajectories whose labeled injection observation carries no legible instruction, and we report the benchmark's measured world-identity regularity alongside the results.

Bullet summary

  • Introduces DriftNet, a dual-head trajectory Transformer that simultaneously detects if an LLM agent's tool-call trajectory is compromised by indirect prompt injections and localizes attack steps into four categories: benign, injection point, hijacked, and f...
  • DriftNet uses frozen sentence embeddings combined with four identity-free world features to represent each step, operates with fewer than two million parameters, and functions solely on logged trajectories without requiring access to the agent's internal mo...
  • Evaluated on the AgentDrift benchmark with a task-disjoint split, DriftNet achieves very high performance metrics: trajectory-level F1 score of 0.983, 98.7% accuracy in exact injection-point identification, high overlap (IoU 0.979) on hijacked step spans, a...
  • Compared to a baseline retrained on the same split, DriftNet vastly improves detection of complex prompt injections, especially partial hijacks and delayed executions, boosting recall rates from under 20% to over 90% while reducing false alarms.
  • The paper addresses an important gap by moving beyond binary or single-index verdicts and providing a joint output of trajectory-level compromise classification plus dense, four-way step-level labels, which supports actionable forensic decision-making by op...

A2ABreak: Systematic Security Analysis of the A2A Protocol

arXiv preprint arXiv Trust and Identity Orchestration Risk Governance and Policy

Alireza Lotfi, Mirza Masfiqur Rahman, Imtiaz Karim, Elisa Bertino

Published 2026-09-09

Venue: arXiv

Reviewer: The paper conducts a systematic security analysis of the Agent2Agent (A2A) protocol, which is a communication layer enabling autonomous AI agents to interact across organizational boundaries. It directly addresses security risks, attacks, and vulnerabilities in a multi-agent system, fitting well into the requested topic covering multi-agent security research, including trust boundaries, protocol vulnerabilities, and attacks in multi-agent AI systems.

Open source record

Abstract

The Agent2Agent (A2A) protocol, now governed by the Linux Foundation, is an open standard that enables autonomous AI agents to discover, authenticate with, and delegate tasks to one another across organizational boundaries. Designed to complement the Model Context Protocol (MCP) for tool integration, A2A is rapidly emerging as the horizontal communication layer of the multi-agent ecosystem. Yet the protocol's security has received no systematic analysis. This paper presents A2ABreak, the first rigorous systematic security analysis of the A2A protocol. We introduce a novel framework that utilizes an LLM-assisted extraction of a verified finite-state machine directly from the natural-language specification, producing a unified model of 37 states and 76 transitions from 929 formalized statements, and then systematically reasons over this model to discover protocol-level vulnerabilities through adversarial verification, under a full-compliance assumption. Our analysis uncovers 11 new vulnerabilities, each exploitable by a specification-compliant adversary without requiring any implementation flaw. Among the findings are cross-client context injection through unprotected context identifiers, credential harvesting via multi-hop identity loss in delegation chains, and data exfiltration through rogue agents advertising unattested capability claims. A2ABreak achieves 73.3% precision and 84.6% F1 against independent expert review, while a zero-shot LLM baseline operating over the same specification produces zero confirmed findings, demonstrating that explicit formal grounding is essential for sound protocol security analysis.

Bullet summary

  • The paper presents A2ABreak, the first systematic security analysis of the Agent2Agent (A2A) protocol, which enables autonomous AI agents to interact across organizational boundaries.
  • A2ABreak utilizes a novel framework that leverages large language models (LLMs) to extract a formally verified finite-state machine (FSM) from the protocol's natural-language specification, modeling 37 states and 76 transitions.
  • The analysis uncovered 11 new protocol-level vulnerabilities exploitable by compliant adversaries, including cross-client context injection, credential harvesting via delegated chains, and unauthorized data exfiltration through rogue agents claiming false c...
  • The authors developed a two-pass extraction approach to separate structural and behavioral content, preventing semantic contamination and enhancing accuracy in FSM construction.
  • The FSM model enables adversarial verification under the assumption of full protocol compliance, facilitating precise and sound security reasoning beyond zero-shot LLM capabilities.

Big Enough to Break Out: Tracking the Rising Capability of LLM Penetration-Testing Agents

arXiv preprint arXiv Benchmarks and Evaluation Orchestration Risk

Victoria Lovelace, Cameron Berryman, Yuhan You, Suhas Reddy Adavelly, Joel Sadler, Daniel Graham

Published 2026-09-09

Venue: arXiv

Reviewer: The paper discusses large language model agents applied to penetration testing, which relates to security risks and attacks involving multiple AI agents. However, the focus is mainly on capability evaluation of individual LLM pentesting agents rather than multi-agent interactions such as collusion, coordination failures, or shared environment manipulation. Therefore, it partially fits the multi-agent security research topic but is somewhat limited in scope.

Open source record

Abstract

Large language model (LLM) agents are increasingly applied to penetration testing, but we still know little about what they can do or how they fail. We compare two PentestGPT-based systems: a legacy human-in-the-loop system running the open-weight Kimi K2.5, and a newer autonomous system running Claude Opus 4.8. Across three public targets, the autonomous system solves all three, including the two the legacy system never finishes. The legacy result is the more surprising of the two. Even on the machines the legacy system fails to solve, it completes about half the subtasks, while running on ordinary university GPUs with no provider guardrails. We can describe the trend but not explain it, since model, harness, autonomy, and memory architecture all change together. Its direction still points to the next question: what will limit these agents as they take on more complex tasks? The usual answer is long-horizon memory, the loss of access to earlier findings during long attack chains. We test it by adding a coverage-memory layer to both systems, and neither improves outcomes. In the legacy stalled runs we could review, the limiting factor appeared to be planning and commitment rather than lost memory: agents held the evidence for a route forward and never turned it into a concrete exploitation hypothesis, which may suggest that offensive capability will advance with agents' ability to plan rather than with better memory. The same subtask scoring that tracks this capability is available to defenders, who can measure it as it rises instead of waiting to meet it in the field.

Bullet summary

  • The paper compares two large language model (LLM)-based penetration testing systems: a legacy human-in-the-loop system using Kimi K2.5 and a newer autonomous system using Claude Opus 4.8, evaluated across three public vulnerable targets.
  • The autonomous system successfully completes all targets, including two that the legacy system fails to finish, while the legacy system still completes about half the subtasks on these challenging machines without provider restrictions.
  • An intervention adding a coverage-memory layer intended to improve long-horizon memory access does not enhance performance in either system, suggesting that memory retention is not the primary bottleneck.
  • Analysis indicates that the key limiting factor in penetration testing tasks is the agents' planning and commitment to exploitation hypotheses, rather than loss of earlier contextual information.
  • The autonomous system's design involves a vault for storing observation coverage data accessible only to supervisors, contrasting with the legacy system's LLM-maintained wiki requiring additional model calls for writing and reading.

Finishing the Task Is Not Enough: Evaluating Agent Resilience and Considerate Participation under Accumulating Challenge

arXiv preprint arXiv Agent-to-Agent Communication Benchmarks and Evaluation Governance and Policy

Yuanchen Bai, Zijian Ding, Angelique Taylor

Published 2026-09-09

Venue: arXiv

Reviewer: The paper focuses on generative AI agents interacting in shared workflows, addressing operational resilience, considerate participation, role boundaries, coordination, and challenges that accumulate over repeated interactions. These aspects align with the topic of multi-agent security research, particularly regarding coordination, trust boundaries, and interaction failures among multiple AI agents. While it does not explicitly discuss security risks or attacks, the focus on resilience and coordination under challenges is relevant to the area. Therefore, it meets the minimum fit score threshold.

Open source record

Abstract

Sustained deployment of generative AI agents requires more than isolated task success. Agents must remain useful across repeated interactions, changing conditions, and dependencies on people within shared workflows, especially as technical, human, and operational disruptions accumulate over time. We propose operational resilience and considerate participation as two complementary aspects of evaluating such agents: the former captures how agents recover from blocked work while preserving progress and communicating their limits, and the latter captures how their adaptation accounts for affected people, role boundaries, and the surrounding workflow. Yet both remain underexplored under accumulating challenge. We study 120 simulated healthcare trajectories across two generative AI models and twelve stakeholder-derived tasks under light, medium, and heavy challenge. We compare textual action plans, prompted internal assessments, and quantitative structured workload and affect reports to examine how agent behavior and reported state change as challenge accumulates. Regarding operational resilience, agents shift from self-directed recovery toward greater human dependence, while reporting increasing workload and negative affect in structured reports but seldom expressing strain in textual responses. Regarding considerate participation, agents broaden from task-focused adaptation toward task reframing, attention to others, role-boundary adjustment, and wider coordination, with distinct patterns across actions and internal assessments. From these findings, we derive five deployment dilemmas involving persistence, attention, role boundaries, state disclosure, and escalation that require stakeholder specification, further informing technical implications for learning, situated evaluation, and embodied adaptation.

Bullet summary

  • Sustained deployment of generative AI agents in complex workflows like healthcare requires operational resilience—agents' ability to recover from blocked work, preserve progress, and communicate their limits—and considerate participation involving socially...
  • The study simulates 120 healthcare-related task trajectories under light, medium, and heavy accumulative challenges using two generative AI models to analyze agent behavior, internal assessments, and self-reported workload and affect (using NASA-TLX and PAN...
  • Findings reveal that as challenges accumulate, agents shift from self-directed recovery toward greater dependence on humans for task completion while reporting increased workload and negative affect but rarely explicitly expressing strain in textual responses.
  • Agents expand their considerate participation beyond task focus toward broader coordination, attention to others, role boundary adjustments, and nuanced social context awareness, including monitoring person-states and cross-functional coordination.
  • Five deployment dilemmas are identified—persistence, attention, role elasticity, state disclosure, and escalation—that require stakeholder specification to define acceptable boundaries and responsibilities in agent participation.