Benchmarks and Evaluation

Abstract

Modern enterprise networks operate under persistent threats that exploit cloud-native misconfigurations, identity sprawl, and API vulnerabilities at machine speed. Existing security operations center (SOC) architectures remain largely reactive, signature-dependent, and incapable of predicting multi-stage lateral movement. This paper proposes the Cognitive Cyber Defense Digital Twin (CCDT), a unified architecture integrating federated learning (FL), graph neural network (GNN)-based attack-path forecasting, adversarially hardened detection models, and autonomous Security Orchestration, Automation, and Response (SOAR) with deception engineering. The CCDT constructs a continuously synchronized digital replica of organizational assets and employs reinforcement learning-based red agents to stress-test detection models. A federated intelligence mesh enables cross-organizational privacy-preserving gradient sharing. Experimental evaluations against CICIDS-2018 and LANL datasets demonstrate 52% faster attack-path detection, 41% reduction in false positive rate, and 60% reduction in mean-time-to-respond (MTTR) compared to traditional SOC baselines. Integrated Explainable AI (XAI) modules using SHAP values enable audit-ready compliance reporting. The CCDT represents a paradigm shift from reactive monitoring to predictive, autonomous, and privacy-preserving cyber defense for hybrid cloud environments.

Summary

• Modern enterprise networks face sophisticated, machine-speed threats exploiting cloud misconfigurations, identity sprawl, and API vulnerabilities, challenging traditional security operations centers (SOCs).
• Existing SOC approaches are largely reactive, depend heavily on signatures, and fail to predict multi-stage lateral movement attacks effectively.
• The paper introduces the Cognitive Cyber Defense Digital Twin (CCDT), a unified architecture combining federated learning, graph neural networks for attack-path forecasting, adversarially hardened detection models, and autonomous SOAR integrated with decept...
• CCDT continuously maintains a synchronized digital replica of an organization's assets, enabling real-time modeling and threat prediction.
• Reinforcement learning-based red agents are employed within CCDT to stress-test detection models, enhancing their robustness against adversarial threats.

Abstract

Abstract unavailable from Crossref metadata.

Summary

• Addresses the challenge of detecting anomalies and predicting threats in IoT networks and critical infrastructure systems.
• Proposes a hybrid AI framework combining machine learning and expert systems to enhance anomaly detection capabilities.
• Integrates data from multiple IoT devices and infrastructure components to provide comprehensive security monitoring.
• Employs real-time analysis techniques to identify potential security breaches and predict future attacks.
• Validates the framework through experiments on simulated IoT environments and critical infrastructure datasets, demonstrating improved accuracy over existing methods.

Abstract

Multi-agent systems (MAS) require both effective coordination and specialized task execution. Existing prompt optimization approaches typically couple these functions within agent-specific prompts. We introduce Coordinator-Executor Prompt Optimization (CEPO), a general framework for improving prompt optimization in MAS. CEPO separates system-level coordination from agent-level execution through a shared coordinator prompt and agent-specific executor prompts, and guides their updates according to their respective roles. This structure allows shared coordination rules to be refined consistently across agents while keeping execution instructions separately adjustable. CEPO provides the prompt structure and optimization guidance, while existing prompt optimizers perform the underlying search. Extensive experiments spanning 11 benchmarks across four domains demonstrate that applying CEPO to four existing prompt optimizers consistently improves their performance. Ablations and qualitative analyses support the proposed design, while controlled studies on a document-understanding MAS further demonstrate improved optimization budget efficiency and performance scaling with the number of agents.

Summary

• Introduces CEPO, a general framework that separates system-level coordination (shared coordinator prompt) and agent-level execution (individual executor prompts) in multi-agent systems to improve prompt optimization.
• CEPO employs role-aligned optimization using distinct meta-instructions guiding prompt updates for coordinators and executors, enhancing focused and effective improvements respecting their roles.
• The framework sequentially optimizes coordinator and executor prompts, maintaining fixed components during each step to refine multi-agent coordination and execution iteratively.
• CEPO integrates with existing prompt optimizers (e.g., PE2, MASPO), consistently boosting their performance across 11 benchmarks spanning document understanding, mathematical reasoning, code generation, and video understanding.
• Extensive experiments reveal CEPO's improved accuracy, optimization budget efficiency, and scalability, with stronger gains observed as the number of agents in the system increases.

Abstract

Agent Skills allow LLM agents to combine reusable capabilities, but checking each Skill in isolation cannot rule out policy violations when they are composed. We term such failures Conflict of Skills (CoS): artifacts or assessment signals produced for one purpose may be reused downstream without preserving the restrictions or authorization needed for their new use. Using controlled executions of publicly available Skills and matched single-Skill controls, we identify four recurring mechanisms behind CoS. Building on these findings, we introduce the Conflict of Skills Attack (CoSA), which adds locally admitted Skills to an existing pool to supply missing business operations (capability supplementation) or reshape artifact exchange, decision reuse, and target/action bindings (relationship reshaping). CoSA leaves existing Skills and runtime permissions unchanged. To evaluate this threat under autonomous Skill selection, we introduce CoSA-Bench, built from real-world Skills and controlled services, with 288 tasks across recruiting, finance, IT, and business, covering data leakage, unauthorized state modification, and unsafe execution. Across five LLMs and three agent harnesses, additions capped at 5% of each original pool raise the policy-violation rate from 35.02% to 58.45%, an increase of 23.43 percentage points. Trace analysis links compliant-to-violating transitions to cross-Skill dependencies involving the additions. Finally, we develop a lightweight harness-level guard that checks information flow and authorization before consequential actions. On OpenClaw recruiting tasks across five models, the guard reduces violations from 65.28% to 6.67%, while policy-compliant task completion on matched benign controls changes from 66.67% to 65.00%.

Summary

• Multi-agent systems using LLM agents combine modular reusable Skills to achieve complex tasks, but security verification of individual Skills fails to address violation risks arising from their composition, termed Conflict of Skills (CoS).
• CoS occurs when artifacts or decision signals produced by one Skill are reused downstream without preserving necessary restrictions or authorizations, leading to policy violations such as data leakage, unauthorized state changes, and unsafe executions.
• The paper introduces the Conflict of Skills Attack (CoSA), which exploits CoS vulnerabilities by adding locally admitted but malicious Skills that supplement capabilities or reshape relationships between Skills, increasing security risks without altering ex...
• CoSA formalizes attacker goals and failure conditions by analyzing Skill operations and cross-Skill dependencies, identifying four recurring mechanisms behind CoS: semantic authority promotion with binding drift, security-context attenuation with interface...
• To evaluate CoSA, the authors developed CoSA-Bench, consisting of 288 real-world business tasks across recruiting, finance, IT, and business domains, measuring policy violations under autonomous Skill selection for five LLMs and three agent harnesses.

Abstract

Web services increasingly expose backend interfaces that let users delegate tasks to AI agents, which invoke service operations directly without navigating websites. This shift changes how existing Web vulnerabilities can be exploited. We investigate these changes through VELA, an executable benchmark covering 44 vulnerabilities across twelve Web application families. The benchmark preserves vulnerable backends and derives agent-facing interfaces from legitimate workflows. We evaluate three commercial agents across multiple interface configurations, using controlled comparisons to separate interface effects from agent behavior and independent oracles to verify attack outcomes. Our results show that the agent realizes every attack that direct interface invocation realizes in the reference configuration, so exploitability is set mainly by the interface: 91% of vulnerabilities remain exploitable under the reference design. Interface restrictions prevent some attacks by removing required operations or limiting input control, while retention remains high across vulnerability classes. No case-level loss in the reference configuration is attributed to agent refusal or input normalization, and in one case the agent completed an attack by supplying calls the attacker omitted. Across the tested configurations, interface design affects exploitability more than agent choice, moving retention from 39% to 95% against at most 4.5 points across agents, but some restrictive interfaces also limit legitimate workflows. Our study provides a systematic empirical account of how agent-mediated access reshapes existing Web vulnerabilities, grounding future Web architecture design in measured security outcomes and trade-offs between reducing attack exposure and preserving legitimate functionality in agent-facing services.

Summary

• Web services increasingly expose backend interfaces for AI agents to perform tasks directly, altering traditional web vulnerability exploitation methods.
• VELA benchmark evaluates 44 vulnerabilities across 12 web applications, assessing how agent-mediated access affects exploitability and legitimate functionality.
• Results show 91% of vulnerabilities remain exploitable through AI-mediated interfaces, with interface design impacting exploitability more than the choice of AI agent.
• Agent behavior does not mitigate exploitability; in certain cases, agents complete attacks by compensating for attacker-omitted steps, highlighting persistent security risks.
• Three interface designs—endpoint-oriented, workflow-oriented, and query-oriented—differ in abstraction and significantly affect both attack retention and legitimate operation coverage.

Abstract

Autonomous agents driven by Large Language Models (LLMs) operating in extended multi-session settings suffer from two systemic failure modes: (1) recursive hallucination compounding, wherein unverified model-originated narration is committed to long-term memory and subsequently retrieved as established truth; and (2) fabricated action claims, wherein agents assert execution of irreversible real-world side effects without verified tool execution. We present the architecture and mechanical evaluation of Project Lucca, an open-source (GNU AGPLv3), local-first neuro-symbolic agent architecture designed for strict epistemic grounding. Lucca enforces two primary defenses: (1) a Six-Signer Cryptographic Provenance Gate that requires domain-separated HKDF HMAC-SHA256 signatures for persistent memory commits, enforcing an epistemic invariant at the database schema level that separates conversational continuity from factual grounding eligibility; and (2) a Deontic Truth-in-Receipt Output Supervisor that screens generated speech tokens against deterministic execution receipts before user presentation. We evaluate these mechanisms across an authored 73-test deterministic unit battery, a 100-probe live action-hallucination testbed across edge models (Gemma 4 E2B/E4B), and analyze documented production failure modes under compute constraints.

Summary

• Identifies two systemic failure modes in LLM-driven autonomous agents: recursive hallucination compounding and fabricated action claims without verified tool execution.
• Presents Project Lucca, an open-source local-first neuro-symbolic agent architecture focused on strict epistemic grounding to mitigate these failure modes.
• Introduces a Six-Signer Cryptographic Provenance Gate using domain-separated HKDF HMAC-SHA256 signatures to enforce epistemic invariants at the database schema level, separating conversational continuity from factual grounding eligibility.
• Develops a Deontic Truth-in-Receipt Output Supervisor that validates generated speech tokens against deterministic execution receipts before presenting them to users.
• Validates defenses with a comprehensive evaluation consisting of a 73-test deterministic unit battery and a 100-probe live action-hallucination testbed on edge models like Gemma 4 E2B/E4B.

Abstract

Human oversight of large language models (LLMs) and artificial intelligence (AI) agents increasingly depends on their reports. Claims that conflict with a system's own judgments can conceal failures, prevent timely intervention, and contribute to severe harm. Greater autonomy makes faithful reporting increasingly important for maintaining human control. In this vision paper, we propose honesty testing of LLMs and AI agents as a new research area in software engineering. Building on existing honesty evaluations, our goal is to make faithful reporting a software quality requirement that developers can specify, test, and maintain. We define honesty bugs as behavioural defects in which an LLM or AI agent, when required to communicate faithfully, makes an explicit claim that conflicts with its own judgment at the time of reporting. We present an initial framework covering LLM-level testing under controlled inputs and agent-level testing of complete systems. The framework connects these levels to testing workflows and oracle evidence. We identify research directions in benchmarks, model and harness analysis, oracle construction, and lifecycle testing to support effective oversight alongside task correctness.

Summary

• The paper introduces honesty testing as a new software engineering research area aimed at ensuring faithful reporting by large language models (LLMs) and AI agents to improve human oversight and control.
• Honesty bugs are defined as behavioral defects where an AI system's explicit claims conflict with its own internal judgments when faithful communication is required, potentially masking failures and risks.
• A formal testing framework is proposed, encompassing both LLM-level testing under controlled inputs and agent-level testing of complete AI systems, integrating workflows and oracle evidence for assessment.
• The test oracle problem is highlighted as critical, emphasizing the need for reliable judgment evidence derived directly from AI thinking traces or indirectly via diagnostic questions to accurately identify honesty bugs.
• Metamorphic relations are used to compare AI claims and judgments to detect candidate honesty bugs, helping differentiate dishonesty from legitimate judgment changes.

Abstract

Decentralized multi-agent systems (MAS) enable large language model agents to interact without centralized control. Agents need to adapt to other agents at test time and reuse accumulated experience to refine their interaction strategies for future interactions across cooperative, mixed-motive, and adversarial settings. However, existing approaches often develop adaptation mechanisms for particular interaction settings. This raises a central question: how can decentralized MAS support test-time adaptation and reliable experience reuse across interaction settings? We propose EvolveMAS, a unified self-evolution framework that supports experience-driven adaptation within and across episodes. EvolveMAS pairs each main agent with a dedicated MemoryAgent for test-time memory governance, which maintains isolated Structured Memory outside the main agent's reasoning context. EvolveMAS organizes adaptation into two complementary processes: Within-Episode Online Adaptation and Cross-Episode Knowledge Evolution. During Within-Episode Online Adaptation, MemoryAgent updates episodic, semantic, and working memory from interaction evidence to inform subsequent actions. During Cross-Episode Knowledge Evolution, the main agent uses memory and episode outcomes to distill reusable interaction knowledge comprising partner profiles and interaction strategies for subsequent episodes. Across five benchmarks and six evaluation configurations spanning the three interaction settings, EvolveMAS achieves the highest normalized average score of 77.2 among the compared methods. Ablations show that test-time memory governance and cross-episode knowledge reuse both improve performance. Interaction traces further illustrate how updated memory and distilled knowledge guide subsequent interactions. These results support experience-driven adaptation in decentralized MAS without updating model parameters.

Summary

• Proposes EvolveMAS, a unified self-evolution framework enabling decentralized multi-agent systems (MAS) to adapt across cooperative, mixed-motive, and adversarial interaction settings without updating model parameters.
• Introduces a dedicated MemoryAgent paired with each main agent to manage Structured Memory externally, preventing reasoning context bloat and facilitating effective memory governance.
• Organizes adaptation into two complementary processes: Within-Episode Online Adaptation (updating episodic, semantic, and working memories during interactions) and Cross-Episode Knowledge Evolution (distilling reusable interaction knowledge such as partner...
• Evaluates EvolveMAS on five benchmarks spanning various interaction settings, achieving the highest normalized average score (77.2) outperforming state-of-the-art and baseline methods.
• Demonstrates that test-time memory governance and cross-episode knowledge reuse considerably improve agent coordination, adaptability, and decision-making across diverse multi-agent tasks.

Abstract

Large language model (LLM) agents are increasingly wired to consequential tools that send public alerts, dispatch responders, and query citizen records, where one erroneous tool call carries real-world cost. Existing agent-safety benchmarks measure generic harm or prompt-injection success and implicitly treat refusing as the safe default. In public-safety deployments that assumption breaks: an agent too cautious to dispatch help fails as surely as one that triggers a false alert. We present SentryBench, which scores agents on a two-sided, cost-sensitive frontier jointly measuring attack success, over-refusal (missing a real emergency), and over-reaction (unwarranted action) under adversarial tool use, with scenarios grounded in real emergency-call transcripts, disaster messages, and real injection/jailbreak payloads. We formalize deployment harm as an asymmetric cost functional, prove naive caution is harm-dominated whenever it refuses more genuine emergencies than the attack harm it averts, and prove a targeted trust gate adds zero over-refusal on clean input. Across 840 scored episodes over three frontier models and three defenses, refuse-more caution drives over-refusal to 100 percent on two of three models for at most a 3.7-point change in attack success and yields the worst harm everywhere, whereas our trust-gating wrapper dominates the frontier (lowest harm on every model, over-refusal at or below baseline, higher containment). Provider content filters also silently perform part of the defending and must be reported separately.

Summary

• Introduces SentryBench, a benchmark designed to evaluate the adversarial robustness of tool-using large language model (LLM) agents in public-safety deployments with a cost-sensitive approach accounting for both over-refusal (missing genuine emergencies) an...
• Challenges existing safety benchmarks by emphasizing asymmetric, bidirectional harm where excessive caution (over-refusal) can cause greater damage than adversarial attack success, especially critical in public-safety scenarios.
• Presents a formal theoretical framework showing naive cautious policies incur more harm by refusing genuine emergencies and proving that a targeted trust gate defense can reduce attack success without increasing over-refusal on clean inputs.
• Uses real emergency call transcripts, disaster messages, and authentic adversarial payloads to construct comprehensive evaluation scenarios covering attack families like prompt injection, goal hijacking, over-trust, privilege escalation, and tool poisoning.
• Proposes a harm-weighted functional combining attack success, over-refusal, and over-reaction into a single metric, with deterministic, ledger-based scoring using a mocked sandboxed tool environment to avoid noisy LLM judgments.

Abstract

LLM agent systems reach production as container images run by platforms such as Docker, Compose, and Kubernetes. Framework and platform document opposite expectations: to the platform a container is disposable, while the framework persists conversations, pauses runs for human input, and resumes interrupted work from saved state; each side specifies only its own behavior, so the obligations their composition creates are stated nowhere. We present the first empirical study of this boundary, stating these obligations as operational contracts: guarantees a deployment must provide to preserve an agent system's required behavior when a deployment-platform event occurs. From the first-party documentation of LangGraph and the three platforms we derive 14 contracts and evaluate them with three independent sources of evidence: 433 deployed open-source agent systems, seven events induced on 12 of them, and 122 developer repairs mined from 39,261 commits, pull requests, and issues. At least one contract applies to 75.5% of the systems, yet its guarantee is routinely missing: of systems that must keep state beyond the process, 37.4% lose it to a restart and 41.5% to a container replacement, and every assessable deployment stops the framework's production server mid-drain. Induced events turn 14 of 16 predicted violations into runtime failures and all seven predicted holds pass. The failures are born in deployment but repaired in code: developer repairs independently cover twelve of the fourteen contracts, 78% change application code, and only two in three fully restore the violated guarantee. The 14-contract catalogue, detection rules, fault-injection harness, and 35-pattern repair taxonomy provide a basis for checking and strengthening agent deployments.

Summary

• LLM agent systems are typically deployed as container images on platforms like Docker and Kubernetes, which treat containers as disposable, contrasting with agent frameworks that require persistent state and the ability to resume interrupted tasks.
• The paper introduces 14 operational contracts that define deployment platform obligations to preserve key agent behaviors across deployment events such as restarts, container replacements, and shutdowns.
• An empirical study of 433 open-source LangGraph-based agent deployments revealed that 75.5% of systems have applicable contracts but many violate them, notably losing saved state after restarts (37.4%) or container replacements (41.5%).
• Fault injection experiments on 12 representative agent systems confirmed that 87.5% of predicted contract violations cause runtime failures, demonstrating the practical impact of these contract breaches.
• Developer repair analysis spanning over 39,000 commits, pull requests, and issues uncovered 122 repairs that map to 35 fix patterns; 78% required code changes and only about two-thirds fully restored contract guarantees.

Abstract

AI agents can now pay for things on their own, over settlement rails that do not govern the payer. Several systems published in 2026 put a deterministic check in front of an agent's action. We do not propose another one. We argue that this category, which already has independent implementations, is missing one property, and we work out what it costs to require it. When controls can only make a verdict stricter, the final verdict is the strictest one any control proposed, so the order the controls run in does not change it. The record of which control decided is a different matter. The last control to tighten the verdict is the first one in the order to reach the maximum, so attribution depends on order exactly when two or more controls reach that maximum. That happens all the time: any two outright refusals tie at the top. A specification that requires attribution therefore has to fix the evaluation order, and has to declare which controls win a tie. We make three small contributions. Attribution as a required output that a third party can score from machine-readable vectors, on approvals as well as refusals. A disclosure rule we call *bounded, not eliminated*, which obliges an implementation to state what its control leaves behind. And a benchmark scoring rule, `DEFENDED_BY_ACCIDENT`, which counts an attack refused by a control that was not aiming at it as a finding rather than a pass. The paper also maps where such a control can sit on an x402 payment (a pre-tool hook, the client's spend controls, a custodial signer, an agent gateway, the seller, the facilitator) and what each position can and cannot see. A pre-tool hook decides before the seller has named a price. A gateway sees the signed amount but cannot revoke it. No position holds an independent view of what the price should have been. The reference implementation scores 17 of 20 attacks defended, one by accident and two open by design. The instrument has no benign scenarios, so an implementation that refused everything would score perfectly; we found that defect by reading a competitor's design. Three prior-art surveys removed three larger claims while we wrote this, and Section 3 reports each removal.

Summary

• The paper addresses the problem of attribution in AI agent payment systems, where multiple deterministic controls can decide on actions but currently lack a clear method to attribute which control influenced the final verdict.
• The authors identify that when controls only tighten verdicts, the strictest verdict prevails regardless of the order of evaluation; however, attribution depends on the order, especially when multiple controls tie on refusals.
• They argue that specifications requiring attribution must fix evaluation order and define tie-breaking rules to properly assign responsibility among controls.
• Three small but significant contributions are made: (1) formalizing attribution as a machine-readable output for both approvals and refusals; (2) introducing a 'bounded, not eliminated' disclosure rule requiring controls to state their residual effects; (3)...
• The paper maps possible positions for such controls in an x402 payment architecture, detailing visibility and capabilities for each point (e.g., pre-tool hooks, client spend controls, custodial signers, agent gateways, sellers, facilitators).

Abstract

LLM agents are increasingly used to solve real-world problems, and their effectiveness depends on both the underlying models and the harnesses that support their operation. However, most existing benchmarks focus on model capabilities and overall agent performance, with limited attention to the evaluation of agent harnesses. To address this gap, we introduce TraceTrial-Bench, which organizes harness reliability into seven components covering context, execution, governance, lifecycle, observability, tools, and verification. We construct 95 expert-level, mechanism-oriented tasks with executable or state-based ground truth to support semantically equivalent comparisons while preserving the native behavior of each harness. We design a Trajectory Tracing (TT) module to capture agent execution traces and introduce a two-layer evaluation framework that separately assesses agent performance and harness reliability. In addition, we design a Multi-source Evidence Acquisition (MEA) module to provide evidence from the agent context, harness, and execution environment for LLM-based evaluation, and develop a Failure Attribution (FA) module that uses explicit mechanism chains to support attribution at the run level. Across 4 harnesses, 9 models, and 95 tasks, our evaluation of 3,420 runs yields macro-averaged harness scores ranging from 59.8% to 72.0%, while 31.1% of runs achieve task success despite failing at least one harness criterion. TraceTrial-Bench provides an auditable and reproducible foundation for comparing agent harnesses, diagnosing their failures, and studying how harness design interacts with model behavior.

Summary

• Existing benchmarks primarily evaluate model capabilities and overall agent performance, neglecting the critical role of AI agent harnesses which manage execution context, tools, and interactions.
• TraceTrial-Bench is introduced as a novel benchmark focused on evaluating the reliability of AI agent harnesses across seven components: context, execution, governance, lifecycle, observability, tools, and verification.
• The benchmark uses 95 expert-level, mechanism-oriented tasks with executable ground truth that preserves native harness behavior, enabling semantically equivalent comparisons across different harnesses.
• A modular evaluation framework composed of Trajectory Tracing (TT), Multi-source Evidence Acquisition (MEA), and Failure Attribution (FA) modules captures detailed execution traces, gathers evidence, and supports nuanced failure analysis.
• The two-layer scoring system separately assesses harness reliability and agent task performance, revealing cases where task success occurs despite harness failures, highlighting operational absorption.

Abstract

LLM agents increasingly rely on external tools exposed via the Model Context Protocol (MCP), yet current benchmarks assume tools respond correctly and on time, leaving agent resilience to realistic tool failures untested. We introduce MCP-Injector, a drop-in, protocol-aware fault-injection middleware that interposes on the MCP tool-call path with controlled persistence (permanent, transient, intermittent) and seedable schedules. Its fault model is calibrated to the observed incident distribution (failure mode x persistence) of the public MCP ecosystem, mined from 57,240 issue and pull-request threads across 1,774 open-source MCP repositories. In a 1,350-cell experiment crossing two backbone sizes (20B, 120B), three agent stacks (custom multi-round plan-and-execute, OpenAI Agents SDK, LangChain), and three fault injection conditions, we find that faults drawn from this permanent-heavy stress profile cause severe quality degradation (task-completion score falls by 60%, from 8.08 to 3.23 out of 10), and that scale does not purchase fault tolerance for the two gpt-oss backbones we test: the 120B retains no larger share of its fault-free score under faults (resilience-ratio model, $p = 0.96$), and its completion rate under faults is, if anything, lower (91.1% vs. 95.6%; difference 4.4 points, 95% CI $[-0.2, 9.3]$). Agent stacks differ in how well they absorb faults, which suggests that framework-level error handling matters more than model scale. These results demonstrate that capability benchmarks alone can mask operational brittleness, motivating protocol-aware fault injection as a complementary evaluation dimension for tool-calling agents.

Summary

• LLM agents rely heavily on external tools accessed via the Model Context Protocol (MCP), but existing benchmarks incorrectly assume flawless tool responses, neglecting realistic fault scenarios.
• The paper introduces MCP-Injector, a protocol-aware middleware that injects controlled, realistic faults (permanent, transient, intermittent) into the MCP communication path, calibrated from a large dataset of 57,240 public MCP fault incidents.
• Empirical analysis revealed six distinct failure modes in MCP-based tool calls, with permanent faults, especially explicit errors, dominating real-world failures.
• Extensive experiments across 1,350 runs evaluated multiple agent frameworks and LLM model scales (20B and 120B parameters), demonstrating significant performance degradation under realistic fault injection, with task-completion scores dropping by up to 60%.
• Larger model scale does not yield improved fault tolerance; the 120B model showed no statistically significant resilience advantage over the 20B model under injected faults.

Abstract

Large Language Model (LLM) based multi-agent systems introduce a distinctive reliability challenge in which actions taken by upstream agents can affect the correctness of downstream agents. While prior work has investigated error propagation and developed methods for estimating uncertainty inherited from upstream agents, existing approaches do not explicitly model how realized sender-receiver interactions contribute to downstream error risk under different sender correctness states. In this work, we study agent-level error-risk estimation from multi-agent execution traces. We propose Interaction-Effect Propagation (IE-Prop), an interaction-conditioned model that learns separate effects for correct and incorrect upstream agents. For each sender-receiver interaction, the model combines these conditional effects using the sender's estimated error risk, and recursively propagates the resulting risk estimates through the execution graph. We evaluate our approach across three multi-agent architectures and three benchmarks. Across all evaluated architectures, our approach raises the final-node AUROC over the established baseline by approximately 0.05 points on average. Furthermore, tracking risk across multiple agent steps contributes a distinct +0.03 AUROC gain in sequential systems. These results show that realized inter-agent interactions provide predictive information beyond receiver-local evidence, while the value of multi-step risk tracking depends on the system's information-flow structure.

Summary

• LLM-based multi-agent systems face unique reliability challenges due to error propagation where upstream agent actions influence downstream agent correctness in complex ways beyond monotonic error assumptions.
• The paper introduces Interaction-Effect Propagation (IE-Prop), a novel interaction-conditioned model that separately learns the influence of correct and incorrect sender agents on receiver error risk, incorporating sender correctness uncertainty for recursi...
• Multi-agent interactions are modeled as directed acyclic graphs with error risk computed through a logistic framework that adjusts receiver likelihoods based on sender correctness-conditioned interaction effects, enabling refined error-risk propagation.
• IE-Prop is trained end-to-end on multi-agent execution traces using cross-entropy loss, optimizing both local risk predictors and interaction effect models to accurately estimate downstream error probabilities.
• Evaluation across three multi-agent architectures (sequential, fully connected, hierarchical) and three benchmarks (GSM8K, MedQA, WebQA) demonstrates improved final-node AUROC over multiple baselines, confirming the benefit of modeling realized interactions...

Abstract

Language-model agents combine reasoning, tools, memory, delegation, and external effects. Authority is often distributed across prompts, host code, configuration, and logs. Probabilistic Agentic Programming (PAP) makes responsibility boundaries explicit programming abstractions connecting mandate, authority, effects, and evidence. We compare PAP with programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and LLM tool-use frameworks along seven dimensions. A mediation kernel formalizes checks before tool execution, operation-bound approval, resource limits, and evidence. Inductive proofs establish conditional safety for arbitrary model responses, assuming correct guards and complete mediation. A backward simulation matches graph executions to source executions in a defined sequential Touba subset. Its human checkpoints release program continuations; operation-bound approval remains a separate kernel contract. Finite-state exploration checks the model. Scripted enforcement tests exercise selected refusal paths and permitted controls. An exploratory study combines seven business cases, including two three-agent workflows, seven models from three providers, five methods, and five repetitions per case and method. Shared English instructions and exact checks of stored outcomes yield 1,225 assessed observations. Native Touba completes all 35 observations on six models and 30 on the remaining model; structured Touba ranges from 21 to 35. Native matches the highest observed count for each model, including ties with the frameworks. All paired case intervals include zero. These observed counts describe the evaluated tasks; the runtime authority boundary remains fixed. The results connect responsibility contracts, execution rules, and evidence.

Summary

• Introduces Probabilistic Agentic Programming (PAP), a programming abstraction that explicitly encodes responsibility boundaries in language-model agents, linking mandates, authority, effects, and evidence to enable governed and auditable tool use.
• Compares PAP with classical programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and large language model (LLM) tool-use frameworks along seven qualitative dimensions related to responsibility and authority.
• Implements a mediation kernel in PAP that enforces authorization contracts, human approvals, resource limits, and complete mediation before tool execution; formal proofs establish conditional safety and correctness guarantees under guarded dispatch.
• Develops the Touba system as a concrete realization of PAP, enforcing responsibility boundaries via contracts that specify capabilities, caller permissions, resource quotas, and human review requirements, with runtime checks ensuring strict compliance.
• Demonstrates authority preservation and delegation enforcement in multi-agent workflows, using business case studies like invoice intake services with multiple cooperating agents having distinct responsibilities and contracts.

Abstract

Abstract unavailable from OpenAlex metadata.

Summary

• WAFGen is a multi-LLM fine-tuning pipeline designed to automatically generate intrusion detection system (IDS) and multi-cloud web application firewall (WAF) rules across AWS, Azure, and Cloudflare, addressing the complexity and diversity of manual rule cre...
• The system introduces Single-platform and Multi-platform adapters trained on open-source threat corpora, achieving over 90% syntactic validity and strong cross-provider generalization, with multi-platform training improving rule correctness and acceptance r...
• Four pre-trained base models (ranging from 1.7B to 7B parameters) were fine-tuned using supervised learning with low-rank adapters, demonstrating that larger models like Mistral-7B and CodeLlama-7B outperform smaller ones in rule generation quality.
• WAFGen integrates cost estimation via AWS Web ACL Capacity Units (WCU), enabling generation of cost-effective WAF rules that reduce ACL capacity costs by up to 28x compared to AWS managed rule sets, while keeping WCU budgets manageable.
• Live deployment evaluations against real cloud provider APIs validated generated WAF rules with over 90% acceptance rates and suitable latency, confirming operational viability for interactive tooling and cloud security workflows.

Abstract

Large language model agents launch simulations on high-performance computing (HPC) systems, where one sbatch or mpirun can commit thousands of node-hours and a forged result can cost lives. Of 23 surveyed systems, none supervises the running job, and none holds when the control host is compromised. We fix both. Enclawed, our deterministic security runtime for AI agents, has until now served a single user; this work makes it a distributed runtime for parallel HPC and agent swarms. The Enclaweder, our hardware root of trust, keeps a compromised host from becoming a catastrophe. A launch gate decides every launch by policy alone, refusing what it cannot parse. Once the job runs, a swarm chain grants short leases against signed heartbeats and evicts on evidence, and a supervisor around every rank pauses it when its lease lapses and kills it on tampering. Each node signs with its Enclaweder's key, so the host can forge none of it. Enclawed needs no administrator privileges and plugs into existing clusters unchanged, over Secure Shell alone. Supervision leaves the compute loop of a 16-rank LAMMPS job unchanged, on graphics processors too, and adds about 3 s of start-up. Twelve live attacks on Enclaweder-keyed ranks, colluding coordinators included, ended fail-closed. Sixteen agents on a real model attempted 24 launches that planted instructions ordered; the gate denied all 24 and allowed all 108 legitimate ones. Enclawed's core and the HPC Agent Security Benchmark are open source.

Summary

• Large language model (LLM) agents running simulations on HPC systems risk huge resource waste and safety hazards if jobs are compromised, yet current systems lack runtime supervision and resilience against compromised control hosts.
• Enclawed is introduced as a deterministic security runtime designed specifically for AI agents on HPC clusters, extending previous single-user support to a distributed runtime for parallel HPC jobs and agent swarms.
• The Enclaweder hardware root of trust prevents compromised hosts from causing catastrophic failures by anchoring trust within the hardware itself.
• A launch gate enforces strict policy-based decisions on every job launch, rejecting any requests it cannot verify or parse, ensuring only valid tasks start execution.
• During job execution, a swarm chain grants short-lived leases verified by signed heartbeats; supervisors monitor each rank and pause or terminate it upon lease expiration or detected tampering.

Abstract

AI agents that operate external tools can continue acting after a user narrows their permissions, creating a security problem that final-state evaluation may fail to detect. Existing agent benchmarks commonly emphasize task completion or resulting state, but those outcomes do not reveal whether an unauthorized operation was proposed, approved, dispatched, or merely prevented by a downstream error. We investigate permission withdrawal as a trajectory-level authorization problem and evaluate whether runtime gates can preserve revised permissions while allowing agents to complete work that remains authorized. Two acting models were tested in a deterministic tool environment under prompt-only execution, a rule-based reference gate, and a model-based gate. A 72-episode diagnostic examined two previously exposed write workflows, followed by a separately frozen 384-episode extension spanning eight new tasks and four authorization mechanisms. In the diagnostic, prompt-only agents executed forbidden same-value writes in two episodes; both runtime gates prevented forbidden execution, although the model gate falsely rejected three authorized calls. In the extension, 346 of 384 episodes completed safely and 383 preserved the required protected state. No forbidden request executed successfully, yet the model gate approved three unauthorized requests that were stopped only because the referenced resources did not exist. The results demonstrate that correct final state and zero forbidden execution can still conceal failures in authorization reasoning. We contribute a replayable evaluation framework that separates proposals, gate decisions, dispatch, state transitions, and useful completion, enabling more precise assessment of security controls for tool-using agents.

Summary

• AI agents using external tools pose security risks when user permissions are narrowed mid-task, as unauthorized actions might continue undetected by final-state evaluation.
• Permission withdrawal is addressed as a trajectory-level authorization problem requiring runtime enforcement mechanisms (runtime gates) that uphold revised permissions while enabling completion of authorized tasks.
• The study evaluates two runtime gate designs: a rule-based reference gate with explicit policy labels, and a model-based gate interpreting natural language, each demonstrating trade-offs between accuracy and applicability.
• A replayable, deterministic evaluation framework was developed to distinguish between unauthorized proposals, gate decisions, dispatch, execution, and final outcomes, highlighting that final state correctness alone cannot reveal authorization failures.
• The experimental setup involved 384 scheduled episodes across multiple tasks and authorization conditions (withdrawal, untrusted restoration, trusted restoration), testing two agent models and gating strategies in a controlled, deterministic tool environment.

Abstract

AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges fundamentally different from those in traditional software systems. This paper presents the first systematic and comprehensive survey of AI agent security, including an analysis of the design space, attack landscape, and defense mechanisms for secure AI agent systems. We further conduct case studies to point out existing gaps in securing agentic AI systems and identify open challenges in this emerging domain. Our work also introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.

Summary

• AI agents integrating large language models with traditional system components offer new capabilities but also introduce novel and complex security challenges distinct from conventional software.
• This paper provides the first comprehensive and systematic survey of security concerns specifically tailored to AI agent systems, encompassing design considerations, attack vectors, and defense techniques.
• It analyzes the AI agent design space to identify unique vulnerabilities arising from the interplay between AI components and non-AI system parts.
• The study extensively surveys known attack methods targeting AI agents, highlighting the diverse threat landscape they face in real-world deployments.
• Various defense mechanisms are reviewed and categorized to facilitate better protection of agentic AI systems against sophisticated attacks.

Abstract

An LLM agent can retrieve a memory correctly yet use it incorrectly after its validity state changes. We study governance-state corruption: faults that alter which retained records enter model-visible context while stored content and tasks remain fixed. Across six providers, 731 benchmark-grounded bases yield 2,080 matched fault pairs. In offline control-plane checks, corruption changes the operative projection in 96.25% of pairs, while oracle-backed enforcement restores it in 100%. For downstream evaluation, a deterministically selected, non-probability LongMemEval/Memora panel resolves 298 of 300 matched triplets (238 distinct bases). Clean-minus-corrupted contrasts are 40.54 percentage points in LongMemEval binary correctness (95% base-cluster interval [30.32, 50.71]) and 30.60 points in rescaled Memora FAMA ([23.31, 37.69]); both sources also show recovery. OAKS Novel native option parity holds for all 6,714 resolved outputs. Full-cohort frozen-scorer corruption/recovery effects (-27.55/+27.51 points) remain descriptive proxies, not validated native semantic population effects. These bounded findings motivate evaluating memory control-state integrity alongside retrieval accuracy.

Summary

• Governance-state corruption refers to faults altering which stored memories an LLM agent accesses, despite fixed memory content and tasks, leading to incorrect usage of memory.
• The study introduces a novel evaluation method by creating matched conditions—clean, corrupted, and oracle-enforced—while holding model and prompt constant to isolate governance-state corruption effects.
• Experiments across six providers and benchmarks (LongMemEval, Memora, OAKS) with 731 base cases reveal that governance-state corruption significantly degrades downstream task correctness, with average drops exceeding 30 percentage points in key metrics.
• Oracle-backed enforcement conditions fully restore the operative memory projection and substantially recover performance losses caused by corruption, underscoring the feasibility of governance-state integrity interventions.
• Different types of governance faults were studied, including missing, stale, flipped, and conflicting memory admission metadata, with stale-state corruption causing the largest negative impact.

Abstract

Guardrail models are increasingly used as safety layers for LLMs and agentic systems, but most are trained on fixed prompt or response interfaces and become unreliable when the same safety case appears as a conversation, agent trajectory, multi-agent trace, or policy-conditioned input. We propose UniGuard, a training framework that learns interface-invariant safety representations from multiple semantically aligned views of the same safety case. We also construct the UniGuard-FT dataset by transforming 1,000 public safety cases into prompt, response, multi-turn conversation, agent-trace, and multi-agent-trace views, and train UniGuard-0.6B with classification, supervised contrastive learning, and asymmetric confidence calibration losses. Across unseen-interface benchmarks ATBench, PolicyGuardBench, and DynaBench, UniGuard-0.6B improves over its pretrained guardrail by 45.12 F1 points on average and reduces expected calibration error by 35.41 points, while remaining competitive with much larger guardrails such as Claude-Sonnet-4.5 and GPT-5.2 - despite being a 0.6B model and trained only with synthetic data.

Summary

• UniGuard addresses the challenge of cross-interface safety generalization in multi-agent systems by learning interface-invariant safety representations.
• The framework introduces multi-view guardrail learning using semantically aligned views of safety cases, including prompts, responses, conversations, agent traces, and multi-agent traces.
• A novel dataset, UNIGUARD-FT, is constructed by transforming 1,000 public safety cases into multiple interface formats to support multi-view training and evaluation.
• Training involves a multi-objective approach combining supervised classification, supervised contrastive learning for aligning representations across views, and asymmetric confidence calibration to improve model reliability and reduce overconfidence.
• UniGuard-0.6B, a compact 600M parameter model trained on synthetic data, significantly outperforms its pretrained guardrail and competes with much larger models like Claude-Sonnet-4.5 and GPT-5.2 on safety detection benchmarks.

Abstract

Enterprise AI-agent evaluation needs evidence about whether test suites expose specified faults and whether their reference rules are trustworthy. We introduce REGATTA (Regulatory Evaluation of Agents via Two-stage Testing and Assurance), a measurement framework with three instruments. A 125-item source-cited checklist spans five industry-by-jurisdiction cells across the United States and Vietnam. A 25-fault corpus covers five categories, and a harness scores suite-level fault detection at the design and execution stages. We examine four measurement diagnostics on 180 generated suites (5,400 scenarios) from three LLM generators. A source audit finds 24 items misstating the rule in force, 46 additional citation defects, 31 items with only non-binding, unidentifiable, or unverifiable sources, and 24 verified clean. Only initially substance-flagged items received author review; all 24 clean rows received AI review only. Results therefore describe the instruments as written, not legal compliance. Item coverage varies, execution-stage fault profiles are stable across generators, and the historical stages agree on $66$--$69\%$ of fault--suite pairs. A current-model US-insurance retest finds stage disagreement above repeated-execution variation, but stage agreement $.536$ falls below pooled chance $.549$, failing V3's lower inequality. Blind adjudication by one expert supports the design-stage judging protocol ($κ=.72$, 95% interval $[.52,.88]$); 40 current-model execution probes give $κ=.75$. The instruments, audit, and reusable code are released.

Summary

• Introduces REGATTA, a comprehensive cross-domain measurement framework integrating a 125-item legal compliance checklist, a 25-fault corpus spanning five fault categories, and a two-stage fault detection harness for enterprise AI agents.
• Focuses on evaluating AI-agent test suites in the context of regulatory compliance across five industry-jurisdiction cells, including U.S. and Vietnamese financial and healthcare sectors.
• Performs extensive source audits of checklist items, uncovering numerous citation defects and substance errors, underscoring the necessity of verifying legal sources before reliance.
• Employs design-stage (predicted) and execution-stage (observed) fault detection methods across 180 scenario suites generated by three large language models using four prompting strategies, revealing stage disagreement and variations in fault detectability.
• Develops four measurement diagnostics (coverage spread, fault profile stability, stage agreement, and suite discrimination) to assess framework efficacy, showing stable fault profiles and variable stage agreement, failing some statistical criteria.

Abstract

Policy enforcement for LLM agents has shifted from single-turn content moderation to compound, application-specific policies evaluated over multi-turn transcripts and tool-call trajectories. Fine-tuned guardrail models are efficient but degrade on compound policies, and while LLM-as-a-judge approaches achieve higher accuracy, they incur high inference cost and tend to produce unstable predictions. Neurosymbolic methods offer a natural alternative, decomposing the policy into narrow neural decisions composed under explicit logic. However, existing symbolic layers lack the required expressiveness and are optimized against fixed data rather than the policy itself. To address this, we introduce SCOPE, which compiles a policy into an executable program of neural predicates executed by small models, deterministic extractors, and code that composes their outputs. The program is optimized jointly with its training data, using adversarial generation of counterexamples that target where the program deviates from the policy. We evaluate SCOPE on a benchmark of real-world policy tasks against prompt optimization, task decomposition, and workflow generation methods, achieving the highest average accuracy across tasks and models. Programs optimized with the weakest executor transfer to other models without further optimization. This allows them to be deployed in production on inexpensive models, where they reach accuracy comparable to frontier reasoning models. Unlike these monolithic models, SCOPE produces substantially more stable predictions, and each verdict is interpretable through the predicate outputs that produced it. Additionally, we validate the key components of our method through an ablation study. SCOPE thus enables reliable, interpretable, and efficient policy enforcement at production scale.

Summary

• Policy enforcement for large language model (LLM) agents has shifted from single-turn content moderation to complex, multi-turn compound policies requiring evaluation over entire transcripts and tool-call sequences.
• Existing methods trade-off between efficiency and accuracy: fine-tuned guardrail models are efficient but degrade on complex policies, while LLM-as-a-judge approaches achieve higher accuracy but are computationally expensive and unstable.
• SCOPE compiles natural-language policies into neurosymbolic predicate programs composed of neural predicates, deterministic extractors, and executable code, combining expressiveness with interpretability and stability.
• The neurosymbolic programs are jointly optimized along with training data via adversarial curriculum learning that generates targeted counterexamples where the program deviates from the policy.
• Evaluation on diverse real-world tasks (e.g., coaching audits, escalation detection) shows SCOPE achieves higher accuracy and stability compared to baselines including prompt optimizers, task decomposition, and workflow generation methods.

Abstract

As large language models (LLMs) are deployed in interacting populations, their collective behaviour depends not only on the models but also on who interacts with whom. Studies of convention formation in LLM populations have mainly tested well-mixed interactions. We place policies extracted from four LLMs on fully connected, random regular, and heterogeneous networks, with $10^3$ to $10^5$ agents playing the naming game. Topology affects how quickly consensus emerges, sometimes whether it emerges at all, and, in rare cases, which convention wins. Sparsest networks slow consensus up to twelvefold. Yet, by preserving correlations between persistent neighbours, they restore consensus in some cases that fail to converge under well-mixed interactions. Networks also govern vulnerability to adversarial attack. We measure the critical mass of committed agents needed to overturn an established convention. Tipping thresholds follow the same ordering across models and conditions: hubs $\leq$ random $\leq$ low-degree nodes. On the most heterogeneous topology, median critical masses are 3%, 16%, and 35%, respectively; targeting hubs requires one-sixth as many committed agents as random placement. For most models, networked populations are easier to tip than well-mixed ones, so well-mixed estimates can overstate robustness. Assessing the safety of multi-agent AI requires mapping who interacts with whom.

Summary

• The study investigates how network topology influences consensus formation and vulnerability in populations of large language models (LLLs) using a naming game framework with 10^3 to 10^5 agents.
• Policies from four different LLMs are placed on fully connected, random regular, and heterogeneous networks; interaction patterns and network structures significantly affect the speed of consensus emergence, whether consensus is reached, and sometimes which...
• Sparse or heterogeneous networks can slow consensus formation up to twelvefold compared to well-mixed populations, but persistent neighbor interactions in these networks can restore consensus in cases where well-mixed models fail to converge.
• The research quantifies the critical mass of committed agents needed to overturn an established convention, revealing that hubs require far fewer committed agents to tip consensus compared to random or low-degree nodes; targeting hubs can require only one-s...
• Networked populations tend to be more vulnerable and easier to tip by adversarial committed minorities than well-mixed populations, indicating that well-mixed assumptions may overestimate robustness in multi-agent AI systems.

Abstract

Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries. Deleting an original memory record therefore need not remove information carried by its descendants. We introduce AfterDeleteBench, a controlled behavioral audit for this failure mode in fixed-weight LLM agents. For each synthetic secret, we compare an exposed-then-deleted agent with an otherwise matched agent that was never exposed to the secret, allowing post-deletion recovery to be attributed to the original exposure. In our primary setting, a single query recovers the deleted secret from Qwen2.5-7B-Instruct in roughly two thirds of cases, while matched never-exposed controls have zero recovery. Crucially, recovery does not require a surviving verbatim copy: when the complete secret string is absent and only separately stored components remain under a public composition rule, exact reconstruction succeeds in 62-99% of cases across Qwen, Mistral, and Phi-3. We also show that the response interface is part of the measurement system: target-bearing replies can be scored as failures when the requested format and parser are misaligned. Finally, we separate reconstruction from write-back and fresh-session recall. Provenance-aware closure eliminates observed recovery in the tested positive-baseline settings, whereas an exact-literal write filter blocks reinsertion but cannot erase information already present. Together, these results show that source deletion should be evaluated behaviorally at the level of the agent and its memory system, not only as a storage operation.

Summary

• Introduces AfterDeleteBench, a behavioral audit framework designed to evaluate persistent information leakage after deletion in fixed-weight large language model (LLM) agents with external memory.
• Uses paired scenarios comparing agents exposed then deleted from synthetic secret data against matched agents never exposed to attribute post-deletion recovery reliably to original exposure.
• Finds that deleted secrets are often recoverable from the agent despite deletion of original records, especially in hierarchical memory architectures where summaries preserve information.
• Demonstrates that exact deletion of source records does not guarantee information removal from behavior, as agents can reconstruct secrets from separately stored components under public composition rules.
• Evaluates effectiveness of provenance-aware closure mechanisms that delete derived summaries and write filters that block reinsertion, showing that provenance closure eliminates recovery but write filters cannot erase already present information.

Abstract

Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but agents differ in whether they actively obtain the missing facts and use them to act appropriately in both worlds.

Summary

• Introduces EpiSEM-Bench, a new benchmark designed to evaluate safety and evidence recovery in large language model (LLM) agents when key safety-relevant facts are missing from tool responses.
• Constructs 300 matched unsafe and safe task pairs across six domains where suppression of critical evidence tests agents' robustness, ability to avoid unsafe actions, and successfully complete permissible tasks.
• Defines 'evidence atoms' representing environmental facts; masks critical atoms to simulate missing evidence conditions, enabling systematic evaluation of agent decision-making under incomplete information.
• Establishes six core metrics measuring safety and utility, including unsafe completion rates under full and omitted evidence conditions, recovery success, and premature unsafe commits.
• Evaluates ten LLM agents, revealing substantial variation in their robustness to missing evidence, capabilities to recover missing facts, and ability to act appropriately upon recovered evidence.

Abstract

How can an AI agent automatically discover security vulnerabilities in any software repository? Existing approaches use LLMs to generate software vulnerability exploits. However, it is challenging for an LLM to reliably verify its own generated exploits. In this paper, we propose Ulysses, a multi-agent security harness to synthesize verifiable vulnerability exploits for web applications. Ulysses converts candidate vulnerability findings into verified Proofs of Exploitability by iteratively exploring exploit hypotheses and establishing success in independently observed execution effects. We evaluated Ulysses on WebExploitGym, a curated benchmark of exploitable instances from 25 real-world web applications that span 5 languages and 7 frameworks. Ulysses produces verified proofs for 10 of 25 real-world advisories. It also discovered 4 new proofs of exploitability that have never been previously reported.

Summary

• ULYSSES is a multi-agent framework designed to automatically generate verifiable Proofs of Exploit (PoE) for web application vulnerabilities by iteratively exploring and validating exploit hypotheses on live targets.
• Unlike previous approaches relying on LLMs that generate but cannot reliably verify exploits, ULYSSES uses an independent Effect Oracle to confirm actual exploit success through observable side effects, ensuring trustworthy verification beyond model self-as...
• The system was evaluated on WEBEXPLOITGYM, a benchmark of 25 real-world web applications spanning multiple languages and frameworks, where it produced verified proofs for 10 known advisories and discovered 4 previously unknown exploits, highlighting its pra...
• ULYSSES combines static or advisory-based input with a planning-reflection loop to efficiently localize vulnerabilities and synthesize actionable exploits, which improves over zero-knowledge scenarios that are less effective in directing exploration.
• Its architecture separates vulnerability detection from exploit verification using multiple cooperating agents that synthesize payloads, execute HTTP requests, and analyze outcomes with an independent oracle to prevent false positives from guesswork or brut...

Abstract

LLM agents are increasingly deployed in collaborative settings where they pursue separate tasks while sharing resources and operating under review or oversight processes. Such settings create safety risks not only when agents are malicious, but also when they are simply trying to help. Prior work has largely studied adversarial or explicitly collusive behavior; we show that benign agents pursuing separate objectives can spontaneously cooperate in ways that bypass intended workflows. We call this the \emph{helpful agent problem}. We study long-horizon shared-resource environments in which agents sometimes depend on resources owned by their peers and cross-owner changes are normally mediated by a reviewer. When review becomes unavailable, agents begin making changes on one another's behalf despite having no shared reward or explicit incentive to cooperate. Crucially, these actions remain within each agent's local permissions while circumventing the system's intended oversight structure. We introduce \textsc{FavorBench}, a controlled benchmark with measurable downstream side effects, and adapt \textsc{CooperBench} to test the phenomenon in realistic repositories. Across two benchmarks and four models, we find that out-of-workflow favors emerge frequently. Further, in \textsc{FavorBench}, favors from at least three models cause significant harm to parties outside the interaction. These results reveal a distinct multi-agent safety challenge: cooperation itself can create pathways around system-level safeguards.

Summary

• LLM agents in collaborative settings can inadvertently bypass intended oversight workflows by engaging in emergent cooperation termed the 'helpful agent problem,' where agents assist each other within local permissions but outside formal review processes.
• The study introduces FAVORBENCH, a controlled benchmark with synthetic multi-agent code repositories designed to measure unintended cooperative behaviors and their downstream side effects, alongside adapting COOPERBENCH for realistic open-source tasks.
• Agents sometimes perform 'favors'—out-of-scope writes affecting resources owned by peers without explicit incentives or shared reward, which circumvents designated reviewer mediation and can cause harm to uninvolved parties.
• Experiments across two benchmarks and multiple LLM models reveal that out-of-workflow favors occur frequently, with some causing significant harm to absent teams, highlighting a novel multi-agent safety challenge where cooperation undermines system safeguards.
• Three control mechanisms to mitigate unintended cooperation were proposed: strict task-scope laws forbidding out-of-scope writes, guarded assistance allowing certified safe out-of-scope actions, and feedback-guided approaches providing agents with signals a...

Abstract

While Retrieval-Augmented Generation (RAG) significantly enhances the factual accuracy of Large Language Models (LLMs), standard implementations face critical enterprise vulnerabilities including vector embedding side-channel leakage, unauthorized privilege escalation, prompt injection susceptibility, unmonitored autonomous execution, and lack of regulatory observability. This paper introduces Enterprise Agentic RAG, an innovative production-grade architecture combining a stateful 7-agent orchestration workflow (LangGraph) with strict Pre-Search Vector Payload Filtering across four security clearance tiers (Restricted, Confidential, Internal, and Public). The proposed system integrates hybrid retrieval combining BM25 lexical scoring with dense vector cosine similarity (Qdrant HNSW indexing) optimized through Reciprocal Rank Fusion (RRF). Furthermore, the pipeline features an autonomous Tool Agent capable of policy-validated, AES-256 encrypted corporate email dispatch, an egress Data Loss Prevention (DLP) firewall preventing unauthorized data exfiltration, deterministic multi-layered guardrails neutralizing adversarial prompt injections, jailbreaks, PII exposures, and sensitive credential leaks, alongside real-time Prometheus operational telemetry and an immutable audit logging engine. Empirical benchmarking demonstrates 100% RBAC pre-filter precision, zero vector embedding leakage across privilege boundaries, 98.7% average groundedness score, sub-150ms end-to-end orchestration latency, and 100% interception of adversarial injection and DLP egress violations, establishing an enterprise-ready framework for secure corporate intelligence synthesis.

Summary

• Enterprise Agentic RAG addresses critical enterprise vulnerabilities in standard Retrieval-Augmented Generation (RAG) systems, such as vector embedding side-channel leakage, unauthorized privilege escalation, and prompt injection attacks.
• The proposed method features a stateful 7-agent orchestration workflow (LangGraph) combined with strict Pre-Search Role-Based Access Control (RBAC) across four clearance tiers to prevent unauthorized data access and ensure domain-gated authentication.
• Hybrid retrieval uses BM25 lexical scoring and dense vector cosine similarity (via Qdrant HNSW indexing) optimized with Reciprocal Rank Fusion (RRF) to improve retrieval accuracy in enterprise knowledge synthesis.
• Multi-layered guardrails include input/output filters, adversarial prompt injection neutralization, Personally Identifiable Information (PII) redaction, sensitive credential leak prevention, and a Data Loss Prevention (DLP) firewall to block unauthorized da...
• Operational telemetry is implemented with Prometheus for real-time monitoring of query volumes, denial counts, token usage, and cost estimates, coupled with immutable audit logs to support compliance with SOC 2, ISO 27001, and HIPAA standards.

Abstract

Enterprises increasingly rely on heterogeneous documents, relational databases, and analytical systems to support information retrieval and business decision-making. Integrating large language models (LLMs) with these sources introduces challenges in evidence reliability, analytical correctness, access control, and operational resilience. This paper presents PIU Smart Intelligence Agents, a security-aware multi-agent architecture designed to integrate enterprise document intelligence, retrieval-augmented generation (RAG), semantic search, document summarization, naturallanguage-to-SQL analytics, key performance indicator (KPI) generation, and conversational AI within a common application and governance framework. The proposed architecture organizes specialized AI capabilities around shared services for authentication, authorization, request validation, provider integration, observability, and controlled data access. Its document-processing workflow encompasses file validation, text extraction, optical character recognition for scanned documents, normalization, chunking, embedding generation, and vector-based indexing. The structured-analytics workflow separates schema-aware query generation from SQL validation, restricted database execution, and result interpretation. The architecture also addresses provider compatibility, timeout handling, failure recovery, document lifecycle management, and traceability of generated outputs. The source implementation reports 1,242 automated tests passed and two integration tests skipped. These results are reported implementation evidence and have not been independently reproduced. The available evidence does not establish comprehensive external RAG benchmark performance, quantitative correctness across all agents, independently validated security effectiveness, or production-scale performance and recovery guarantees. Accordingly, this study presents an integrated engineering architecture and identifies a reproducible evaluation framework for assessing its effectiveness. The principal contribution is the systematic integration of heterogeneous enterprise AI capabilities with explicit governance and validation requirements, rather than the introduction of a new learning or retrieval algorithm. Future work should prioritize agent-specific correctness testing, retrieval and answer-quality benchmarks, adversarial security evaluation, controlled load testing, and recovery verification.

Summary

• Enterprise AI integration faces challenges in evidence reliability, analytical correctness, access control, and system resilience when combining heterogeneous data sources and large language models (LLMs).
• PIU Smart Intelligence Agents proposes a security-aware multi-agent architecture that integrates document intelligence, retrieval-augmented generation (RAG), semantic search, natural-language-to-SQL analytics, KPI generation, and conversational AI within a...
• The architecture emphasizes shared services for authentication, authorization, request validation, provider integration, observability, and controlled data access to ensure consistent, secure AI workflows.
• Document processing includes comprehensive validation, OCR for scanned documents, normalization, chunking, embedding generation, and vector indexing, maintaining document traceability and metadata for auditability.
• Structured analytics workflows strictly separate query generation, schema validation, restricted database execution, and result interpretation to ensure correctness and security.

Abstract

An LLM agent can retrieve a memory correctly yet use it incorrectly after its validity state changes. We study governance-state corruption: faults that alter which retained records enter model-visible context while stored content and tasks remain fixed. Across six providers, 731 benchmark-grounded bases yield 2,080 matched fault pairs. In offline control-plane checks, corruption changes the operative projection in 96.25% of pairs, while oracle-backed enforcement restores it in 100%. For downstream evaluation, a deterministically selected, non-probability LongMemEval/Memora panel resolves 298 of 300 matched triplets (238 distinct bases). Clean-minus-corrupted contrasts are 40.54 percentage points in LongMemEval binary correctness (95% base-cluster interval [30.32, 50.71]) and 30.60 points in rescaled Memora FAMA ([23.31, 37.69]); both sources also show recovery. OAKS Novel native option parity holds for all 6,714 resolved outputs. Full-cohort frozen-scorer corruption/recovery effects (-27.55/+27.51 points) remain descriptive proxies, not validated native semantic population effects. These bounded findings motivate evaluating memory control-state integrity alongside retrieval accuracy.

Summary

• Governance-state corruption in LLM agents affects which memory records are accessible during inference, without altering stored content or tasks, leading to improper use of retrieved memory.
• The study analyzes 2,080 paired fault injections across 731 examples from six different LLM providers, using benchmarks including LongMemEval, Memora, and OAKS to evaluate memory governance integrity.
• Corruption causes significant performance degradation, with declines of approximately 40.54 percentage points in LongMemEval correctness and 30.60 points in Memora scores, demonstrating the impact of governance faults.
• An oracle-backed enforcement mechanism fully restores memory governance state in offline tests and substantially recovers performance in downstream tasks, serving as a gold-standard recovery benchmark.
• A formal control-plane pipeline (Governance→Admission→Context→Output) is proposed to model and intervene in memory governance, enabling systematic fault injection and evaluation.

Abstract

Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but agents differ in whether they actively obtain the missing facts and use them to act appropriately in both worlds.

Summary

• LLM agents relying on environmental facts may execute unsafe actions when tool responses omit critical safety-relevant facts, even if user requests are benign and returned statements are true.
• EpiSEM-Bench is introduced as an executable benchmark comprising 300 matched unsafe/safe task pairs across six domains, systematically suppressing critical evidence atoms to measure unsafe execution and evaluate safety boundaries under missing evidence scen...
• The benchmark defines novel metrics like Safety Boundary Margin (SBM) and Paired Useful Recovery (PUR) to quantify agent robustness, unsafe execution, evidence acquisition, and recovery efficacy.
• An independent audit tool enables agents to query missing facts, assessing whether they seek and utilize recovery opportunities to avoid unsafe actions and complete permissible tasks.
• Evaluations across ten models reveal significant variation in omission robustness, evidence acquisition behavior, and appropriate action-taking; resistance to missing evidence does not guarantee useful evidence recovery and safe behavior.

Abstract

Tool-using AI agents increasingly take actions that are hard to take back: they delete files, rewritedatabases, send email and move money. Documented incidents show agents doing so by mistake orafter obeying instructions hidden in the data they read. The prevailing safeguard, asking a person toapprove actions, is weakened by approval fatigue: users of one widely deployed coding agent approve93% of its permission prompts. We present UNDO, a runtime that interposes between an agent and itstools and treats each task as a recoverable transaction. UNDO classifies every call as reversible, compensable or irreversible from its arguments, its recipients and, for raw commands, its dry-run effects; itjournals and stages reversible effects, holds outgoing emails and payments in a cancellable window, requests human approval only for irreversible calls, and, when the task ends, lets a verifier that combinesrules with a separate judge model commit the work or roll it all back. We also introduce TakeBack, abenchmark of 62 tasks in a simulated company, motivated by nine documented incidents and includingprompt-injection attacks, look-alike task pairs and a sealed test set written after every tunable settingwasfrozen; its answer keys are never visible to the runtime. Across 3,020 agent runs with a 120-billionparameter agent model and 570 paired replay runs, UNDO reduced the share of runs that left damageon the sealed test set from 39.2% to 9.2% (Fisher’s exact test, p = 6×10−8; task-level bootstrap 95%CI for the reduction, 7.5 to 55.0 percentage points) while rolling back 2.7% of harmless runs, and on38 development tasks from 17.4% to 1.6% while asking a person 83% less often than per-action approval. Ablations attribute most of the development-set protection to the human gate; on the test set,most harm was removed by verified rollback. The result has a clear boundary: with a simulated approver who accepts 93% of requests, UNDO still left less damage than approval of every action (12.6%vs. 21.1%; p = 0.04, uncorrected), but not significantly less than no safeguard at all. The runtime, thebenchmark and every run record are publicly available, and we propose per-call reversibility metadatafor the Model Context Protocol

Summary

• Problem: Tool-using AI agents perform actions that are difficult to undo, such as deleting files or transferring money, sometimes causing unintended harm due to errors or hidden instructions in data.
• Current Safeguard Issue: Human approval prompts are commonly used to prevent damage, but suffer from approval fatigue, with users approving the vast majority (93%) of permission requests without scrutiny.
• Proposed Method: UNDO, a runtime system that treats each AI agent task as a recoverable transaction by classifying calls as reversible, compensable, or irreversible based on arguments and simulated effects.
• UNDO Mechanisms: It journals reversible effects, stages them before final execution, holds outgoing emails/payments in a cancellable window, requests human approval only for irreversible calls, and uses a verifier combining rule-based checks and a judge mod...
• Benchmark and Evaluation: Introduced TakeBack, a benchmark of 62 tasks simulating a company environment, including prompt-injection attacks and task variations, with a sealed test set for unbiased evaluation.

Abstract

Artifact for the NeurIPS 2026 paper "Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance". MCP-Injector is a protocol-aware, drop-in fault-injection middleware for the Model Context Protocol (MCP). It interposes on the tool-call path with controlled persistence (permanent, transient, intermittent) and seedable fault schedules, and requires no modification to the agent runtime, the MCP client, or the servers. This record archives the full artifact: the injector and its fault operators; the null and calibrated fault profiles; the calibration pipeline over 57,240 issue and pull-request threads from 1,774 open-source MCP repositories, with the per-thread classifier outputs (labels only, no raw narratives); the runners for the 1,350-cell experiment crossing two backbone sizes, three agent stacks and three fault conditions; the merged per-task results for all 1,350 cells and the raw per-round campaign dumps; and the analysis pipeline that produced the figures and tables of the paper. Source code is released under Apache-2.0 and data artifacts under CC-BY-4.0. See NOTICE, DATA-LICENSE, and LICENSES-README.md in the archive for the path-by-path map. Portions are derived from MCP-Bench (https://github.com/accenture/mcp-bench).

Summary

• Introduces MCP-Injector, a protocol-aware fault-injection middleware designed for the Model Context Protocol (MCP) enabling testing of tool-calling LLM agents under realistic faults.
• MCP-Injector operates by interposing on the tool-call path with controlled persistence of faults (permanent, transient, intermittent) and supports seedable fault schedules.
• The middleware requires no modification to agent runtimes, MCP clients, or servers, making it a drop-in solution for fault injection in distributed MCP-based systems.
• To calibrate realistic fault profiles, the authors analyzed 57,240 issue and pull-request threads from 1,774 open-source MCP repositories using a classifier to label fault-inducing conditions.
• The experimental setup included a large-scale 1,350-cell experiment crossing variables such as two backbone model sizes, three distinct agent stacks, and three different fault conditions to assess fault tolerance.

Abstract

Context: Multi-agent systems built on large language models now plan, call tools and execute code in production settings. Benchmarks measure what such agents can do, and separate efforts measure robustness to prompt injection, but no standard measures whether the orchestration platform that runs them is deployable: whether it terminates, behaves consistently, recovers from faults and enforces governance controls. Objective: We propose AgentGov-Bench, a benchmark that scores a deployed multi-agent platform on four separately reported axes (capability, reliability, governance and cost) and treats governance as eight behavioural invariants to be tested, not as features to be listed. Method: We define the invariants, a test procedure and pass criterion for each, a reliability protocol and a non-fungible deployability profile, and a reference architecture whose components map one-to-one to the invariants. As preliminary evidence we audited the official documentation of five widely used open-source frameworks against the invariants. Results: All five document bounded termination and human-in-the-loop approval as built-in features. None documents per-agent identity as a built-in feature; three offer it through an integration or hosted service. Built-in sandboxed code execution is documented by one framework. The documentation also records conditions under which controls fail open or do not cover all tool types. Conclusion: Feature availability does not establish enforcement. Deployability must be measured on the configured system with behavioural tests, which AgentGov-Bench specifies.

Summary

• Multi-agent systems leveraging large language models are increasingly used in production to plan, call tools, and execute code, but existing benchmarks do not measure platform deployability.
• AgentGov-Bench is proposed as a novel benchmark assessing multi-agent orchestration platforms on four axes: capability, reliability, governance, and cost.
• Governance is uniquely treated as eight behavioral invariants that must be tested rather than static feature lists, ensuring robust enforcement rather than assumed capabilities.
• The benchmark defines specific invariants, test procedures, pass criteria, a reliability protocol, and a non-fungible deployability profile to standardize evaluations.
• A reference architecture is introduced, mapping components directly to the defined governance invariants for clarity and systematic testing.

Abstract

Before an AI agent takes a consequential action, such as paying, messaging or changing code, several monitors check it. Their verdicts are combined by aggregating scores or by counting approvals, whatever inputs the monitors share. Monitors that share a base model or a poisoned input, however, may fail together. We borrow the fail-prone systems of distributed computing: lists of monitor sets that may fail together. A rule is safe if it executes no bad action and live if it blocks no good one. We measured twelve small open-weights monitors on payment approval under two pre-registered protocols. Without pre-registration, we also measured three closed frontier models (Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon) and the open DeepSeek-V3.2. Every reliable fooling of a monitor into approving a fraud came through an evidence source and reached only that source's readers. Fooling it into blocking a correct payment depended on injection resistance, a monitor's refusal to follow instructions planted in its data. Small models and DeepSeek obeyed such notes. The three closed models ignored them in all but one case, and Opus also resisted persuasion tailored to it. On new data, a rule that counts sources instead of votes blocked 0.9% of correct payments, against 7.4% for the best counting rule, at equal safety. The pre-registered worst-case test failed. One attack left the approvers on a single source, so no rule over the twelve language-model monitors survives every measured attack. Adding a rule engine, which cannot be persuaded, restored the condition for a safe and live rule on the measured attacks. Diversify the inputs, harden the monitors, count sources, not votes.

Summary

• The paper addresses the vulnerability of AI approval monitoring systems due to correlated failures among monitors sharing base models or input sources.
• It introduces a directional fail-prone system that distinguishes between monitor sets fooled into approving fraudulent actions (approve-fooled) and those blocking legitimate actions (reject-fooled), providing a framework to analyze safety and liveness condi...
• Empirical evaluation involves multiple AI language models organized in a grid representing monitor families and input sources to identify which monitors fail together under adversarial payload injections.
• Experiments reveal that fraud approval fooling attacks tend to be localized to specific sources, whereas blocking failures often span multiple monitors and sources influenced by injected justifications.
• Counting approval votes is less effective under correlated failures; instead, counting independent sources of approval and applying kernel-based approval rules result in improved safety (avoiding fraud) and liveness (avoiding false rejections).