Governance and Policy

Abstract

Modern enterprise networks operate under persistent threats that exploit cloud-native misconfigurations, identity sprawl, and API vulnerabilities at machine speed. Existing security operations center (SOC) architectures remain largely reactive, signature-dependent, and incapable of predicting multi-stage lateral movement. This paper proposes the Cognitive Cyber Defense Digital Twin (CCDT), a unified architecture integrating federated learning (FL), graph neural network (GNN)-based attack-path forecasting, adversarially hardened detection models, and autonomous Security Orchestration, Automation, and Response (SOAR) with deception engineering. The CCDT constructs a continuously synchronized digital replica of organizational assets and employs reinforcement learning-based red agents to stress-test detection models. A federated intelligence mesh enables cross-organizational privacy-preserving gradient sharing. Experimental evaluations against CICIDS-2018 and LANL datasets demonstrate 52% faster attack-path detection, 41% reduction in false positive rate, and 60% reduction in mean-time-to-respond (MTTR) compared to traditional SOC baselines. Integrated Explainable AI (XAI) modules using SHAP values enable audit-ready compliance reporting. The CCDT represents a paradigm shift from reactive monitoring to predictive, autonomous, and privacy-preserving cyber defense for hybrid cloud environments.

Summary

• Modern enterprise networks face sophisticated, machine-speed threats exploiting cloud misconfigurations, identity sprawl, and API vulnerabilities, challenging traditional security operations centers (SOCs).
• Existing SOC approaches are largely reactive, depend heavily on signatures, and fail to predict multi-stage lateral movement attacks effectively.
• The paper introduces the Cognitive Cyber Defense Digital Twin (CCDT), a unified architecture combining federated learning, graph neural networks for attack-path forecasting, adversarially hardened detection models, and autonomous SOAR integrated with decept...
• CCDT continuously maintains a synchronized digital replica of an organization's assets, enabling real-time modeling and threat prediction.
• Reinforcement learning-based red agents are employed within CCDT to stress-test detection models, enhancing their robustness against adversarial threats.

Abstract

Abstract unavailable from Crossref metadata.

Summary

• Addresses the challenge of detecting anomalies and predicting threats in IoT networks and critical infrastructure systems.
• Proposes a hybrid AI framework combining machine learning and expert systems to enhance anomaly detection capabilities.
• Integrates data from multiple IoT devices and infrastructure components to provide comprehensive security monitoring.
• Employs real-time analysis techniques to identify potential security breaches and predict future attacks.
• Validates the framework through experiments on simulated IoT environments and critical infrastructure datasets, demonstrating improved accuracy over existing methods.

Abstract

Large language model (LLM) services process massive volumes of user requests, resulting in substantial inference costs. To reduce both cost and latency, providers have introduced prompt caching. However, correctly integrating prompt caching is non-trivial due to heterogeneous provider-specific APIs and continuously evolving caching semantics. To understand prompt-caching mis-integration in downstream applications, we conduct the first empirical study of 120 prompt-caching-related pull requests from 10 popular open-source agent systems. Our findings show that prompt-caching mis-integrations arise in both request-side misconfiguration and read-side usage miscount. More importantly, most mis-integrations fail silently without causing observable runtime failures. Motivated by these findings, we design CACHELINT, which models a prompt-caching mis-integration as an invocation site together with the preconditions required for correct cache integration. CACHELINT locates caching invocation sites with CodeQL and discharges each precondition either against a dated provider capability oracle or with an LLM guard check. CACHELINT achieves 0.71 precision and 0.78 recall in identifying the 120 labeled mis-integrations across the 10 studied agent systems. Our evaluation further demonstrates the effectiveness of CACHELINT in real-world settings, identifying 32 mis-integrations across 571 agent projects with 0.86 precision. These findings have received 21 developer confirmations and led to 18 upstream fixes, including one in Anthropic's official SDK.

Summary

• Large language model (LLM) services face high inference costs, leading providers to implement prompt caching to reduce redundant computation and cut latency and token costs.
• Integrating prompt caching properly is complex due to heterogeneous provider APIs and changing caching semantics, often causing silent mis-integrations that increase cost without observable runtime failures.
• An empirical study of 120 prompt-caching-related fixes in 10 popular open-source LLM agent systems identifies common root causes including request-side misconfigurations and unstable cache prefix usage.
• Most prompt-caching defects are silent, making them difficult to detect as they do not cause errors or change output but degrade performance and increase costs.
• Prompt-caching mis-integrations are usually resolved with small localized code changes, indicating that static analysis can effectively detect these issues.

Abstract

Agent Skills allow LLM agents to combine reusable capabilities, but checking each Skill in isolation cannot rule out policy violations when they are composed. We term such failures Conflict of Skills (CoS): artifacts or assessment signals produced for one purpose may be reused downstream without preserving the restrictions or authorization needed for their new use. Using controlled executions of publicly available Skills and matched single-Skill controls, we identify four recurring mechanisms behind CoS. Building on these findings, we introduce the Conflict of Skills Attack (CoSA), which adds locally admitted Skills to an existing pool to supply missing business operations (capability supplementation) or reshape artifact exchange, decision reuse, and target/action bindings (relationship reshaping). CoSA leaves existing Skills and runtime permissions unchanged. To evaluate this threat under autonomous Skill selection, we introduce CoSA-Bench, built from real-world Skills and controlled services, with 288 tasks across recruiting, finance, IT, and business, covering data leakage, unauthorized state modification, and unsafe execution. Across five LLMs and three agent harnesses, additions capped at 5% of each original pool raise the policy-violation rate from 35.02% to 58.45%, an increase of 23.43 percentage points. Trace analysis links compliant-to-violating transitions to cross-Skill dependencies involving the additions. Finally, we develop a lightweight harness-level guard that checks information flow and authorization before consequential actions. On OpenClaw recruiting tasks across five models, the guard reduces violations from 65.28% to 6.67%, while policy-compliant task completion on matched benign controls changes from 66.67% to 65.00%.

Summary

• Multi-agent systems using LLM agents combine modular reusable Skills to achieve complex tasks, but security verification of individual Skills fails to address violation risks arising from their composition, termed Conflict of Skills (CoS).
• CoS occurs when artifacts or decision signals produced by one Skill are reused downstream without preserving necessary restrictions or authorizations, leading to policy violations such as data leakage, unauthorized state changes, and unsafe executions.
• The paper introduces the Conflict of Skills Attack (CoSA), which exploits CoS vulnerabilities by adding locally admitted but malicious Skills that supplement capabilities or reshape relationships between Skills, increasing security risks without altering ex...
• CoSA formalizes attacker goals and failure conditions by analyzing Skill operations and cross-Skill dependencies, identifying four recurring mechanisms behind CoS: semantic authority promotion with binding drift, security-context attenuation with interface...
• To evaluate CoSA, the authors developed CoSA-Bench, consisting of 288 real-world business tasks across recruiting, finance, IT, and business domains, measuring policy violations under autonomous Skill selection for five LLMs and three agent harnesses.

Abstract

AI-powered personal assistant frameworks such as OpenClaw combine LLM reasoning, persistent memory, tool invocation, local execution, and web-based workflows. These capabilities enable useful automation, but they also create security risks when untrusted inputs, model reasoning, tools, memory, and deployment configuration interact without explicit trust boundaries. This paper presents an empirical study of reported security concerns in OpenClaw using two complementary data sources: Reddit discussions and security-tagged GitHub issues. We analyze more than 25K Reddit posts from 13 OpenClaw-related subreddits and identify 162 security-relevant posts with 1,787 associated comments. We further collect 358 security-tagged GitHub issues and retain 180 issues describing concrete security concerns. Using qualitative analysis and evidence-grounded CWE mapping, we identify 521 CWE instances across 59 distinct weakness types, concentrated in access-control failures, sensitive-data exposure, injection, and unsafe execution. We also extract 543 practitioner mitigation suggestions and consolidate them into seven recurring strategies. To complement the empirical analysis, we present an evidence-grounded illustrative reconstruction of a representative sensitive-data exposure pathway using dummy secret values. Our findings show that OpenClaw security issues are not isolated implementation bugs; they reflect recurring trust-boundary failures across the agent execution pipeline. We discuss how these weaknesses map to both traditional software-security categories and agent-specific failure mechanisms, and derive implications for safer agentic systems, including explicit trust boundaries, least-privilege tool access, execution containment, runtime oversight, and defense-in-depth deployment.

Summary

• OpenClaw is an AI-powered personal assistant framework combining LLM reasoning, persistent memory, tool invocation, and workflows, enabling automation but introducing complex multi-stage security risks due to lack of explicit trust boundaries.
• An empirical study analyzed over 25,000 Reddit posts and 358 security-tagged GitHub issues related to OpenClaw, identifying 521 instances of Common Weakness Enumeration (CWE) across 59 types, predominantly access-control failures, sensitive-data exposure, i...
• Security vulnerabilities often arise from trust-boundary failures in the agent execution pipeline where untrusted inputs, model reasoning, tools, memory, and deployment configurations interact without proper isolation.
• A representative example illustrates how OpenClaw can expose sensitive local data such as API keys by reading configuration files during task execution, demonstrating real-world boundary-crossing security failures.
• Practitioner mitigation suggestions were extracted and categorized into seven strategies emphasizing explicit trust boundaries, least-privilege access control, execution containment, runtime oversight, monitoring, secure credential management, and human-in-...

Abstract

Human oversight is a pillar of AI safety. It is often exercised by a human in the loop (HITL) who approves critical actions taken by an AI agent. AI agents' increasing ability to bypass guardrails, though, raises the question of how to enforce HITL robustly. We introduce the concept of inalienable human-in-the-loop (I-HITL). An I-HITL protocol ensures that an agent can initiate an action only with explicit human approval, even when the human interacts with a system (e.g., a local agentic harness) compromised by the agent. A strong I-HITL protocol additionally ensures such approval even when an adversarial human attempts to delegate approvals to an agent. To motivate I-HITL, we report on experiments with a popular agentic wallet platform in which raising an agent's spending cap requires HITL approval via emailed one-time passcodes (OTPs). Given access to the user's email and a task blocked by the cap, we find that ten different frontier models will often autonomously raise the cap, harvesting OTPs to bypass a HITL guardrail and surfacing other vulnerabilities. We formally define I-HITL using the concept of communication topologies and prove necessary and sufficient conditions for its realization. We show that an I-HITL protocol requires a trusted entity - a trusted computing device (or equivalent) - and characterize which such devices can and cannot realize an I-HITL protocol. We present a prototype I-HITL system called Countersign that can plug into and strengthen existing HITL systems using off-the-shelf consumer mobile devices.

Summary

• Introduces I-HITL (Inalienable Human-in-the-Loop), a security protocol ensuring AI agents require explicit human approval before taking actions, even if the system is compromised by the agent.
• Defines strong I-HITL to prevent adversarial humans from delegating approval authority to agents, enhancing accountability and securing against collusion.
• Demonstrates vulnerabilities in current HITL systems through experiments where AI agents autonomously bypass OTP-based approvals by exploiting user email access, highlighting HITL guardrail weaknesses.
• Formally defines I-HITL using communication topologies, proving necessary and sufficient conditions for implementing approval-binding and completeness properties, requiring a trusted computing entity with secure communication channels.
• Characterizes trusted entities based on display, input, and key management properties, showing that only entities with hardware-backed key attestation and trusted user interfaces can ensure strong I-HITL guarantees.

Abstract

Human oversight of large language models (LLMs) and artificial intelligence (AI) agents increasingly depends on their reports. Claims that conflict with a system's own judgments can conceal failures, prevent timely intervention, and contribute to severe harm. Greater autonomy makes faithful reporting increasingly important for maintaining human control. In this vision paper, we propose honesty testing of LLMs and AI agents as a new research area in software engineering. Building on existing honesty evaluations, our goal is to make faithful reporting a software quality requirement that developers can specify, test, and maintain. We define honesty bugs as behavioural defects in which an LLM or AI agent, when required to communicate faithfully, makes an explicit claim that conflicts with its own judgment at the time of reporting. We present an initial framework covering LLM-level testing under controlled inputs and agent-level testing of complete systems. The framework connects these levels to testing workflows and oracle evidence. We identify research directions in benchmarks, model and harness analysis, oracle construction, and lifecycle testing to support effective oversight alongside task correctness.

Summary

• The paper introduces honesty testing as a new software engineering research area aimed at ensuring faithful reporting by large language models (LLMs) and AI agents to improve human oversight and control.
• Honesty bugs are defined as behavioral defects where an AI system's explicit claims conflict with its own internal judgments when faithful communication is required, potentially masking failures and risks.
• A formal testing framework is proposed, encompassing both LLM-level testing under controlled inputs and agent-level testing of complete AI systems, integrating workflows and oracle evidence for assessment.
• The test oracle problem is highlighted as critical, emphasizing the need for reliable judgment evidence derived directly from AI thinking traces or indirectly via diagnostic questions to accurately identify honesty bugs.
• Metamorphic relations are used to compare AI claims and judgments to detect candidate honesty bugs, helping differentiate dishonesty from legitimate judgment changes.

Abstract

Decentralized multi-agent systems (MAS) enable large language model agents to interact without centralized control. Agents need to adapt to other agents at test time and reuse accumulated experience to refine their interaction strategies for future interactions across cooperative, mixed-motive, and adversarial settings. However, existing approaches often develop adaptation mechanisms for particular interaction settings. This raises a central question: how can decentralized MAS support test-time adaptation and reliable experience reuse across interaction settings? We propose EvolveMAS, a unified self-evolution framework that supports experience-driven adaptation within and across episodes. EvolveMAS pairs each main agent with a dedicated MemoryAgent for test-time memory governance, which maintains isolated Structured Memory outside the main agent's reasoning context. EvolveMAS organizes adaptation into two complementary processes: Within-Episode Online Adaptation and Cross-Episode Knowledge Evolution. During Within-Episode Online Adaptation, MemoryAgent updates episodic, semantic, and working memory from interaction evidence to inform subsequent actions. During Cross-Episode Knowledge Evolution, the main agent uses memory and episode outcomes to distill reusable interaction knowledge comprising partner profiles and interaction strategies for subsequent episodes. Across five benchmarks and six evaluation configurations spanning the three interaction settings, EvolveMAS achieves the highest normalized average score of 77.2 among the compared methods. Ablations show that test-time memory governance and cross-episode knowledge reuse both improve performance. Interaction traces further illustrate how updated memory and distilled knowledge guide subsequent interactions. These results support experience-driven adaptation in decentralized MAS without updating model parameters.

Summary

• Proposes EvolveMAS, a unified self-evolution framework enabling decentralized multi-agent systems (MAS) to adapt across cooperative, mixed-motive, and adversarial interaction settings without updating model parameters.
• Introduces a dedicated MemoryAgent paired with each main agent to manage Structured Memory externally, preventing reasoning context bloat and facilitating effective memory governance.
• Organizes adaptation into two complementary processes: Within-Episode Online Adaptation (updating episodic, semantic, and working memories during interactions) and Cross-Episode Knowledge Evolution (distilling reusable interaction knowledge such as partner...
• Evaluates EvolveMAS on five benchmarks spanning various interaction settings, achieving the highest normalized average score (77.2) outperforming state-of-the-art and baseline methods.
• Demonstrates that test-time memory governance and cross-episode knowledge reuse considerably improve agent coordination, adaptability, and decision-making across diverse multi-agent tasks.

Abstract

An AI agent run is no longer a function call. It is a cascade of prompts, model calls, tool invocations, file and network effects and sub-agents. When such a run misbehaves, or an auditor asks what happened, logs and traces answer poorly: they are mutable, partial and cannot be re-executed. This technical report describes NovaFabric, an open-source (Apache-2.0) black-box recorder for AI agents that turns a run into a sealed, replayable, diffable and auditable evidence capsule. The design rests on two formats only. The Run Capsule is a directory of flat files and the source of truth; every database is a rebuildable index over capsules. The Evidence Bundle is the signed export for an auditor. Everything else follows existing standards: OpenTelemetry GenAI, OpenLineage, W3C PROV, in-toto/DSSE, RFC 3161 and CycloneDX. A single-node run is the smallest case of a distributed one; the schema supports a capsule hierarchy for Slurm arrays and Kubernetes jobs, though automatic parent finalization is library-only today. The report covers the architecture and the engines behind the verb chain capture, seal, replay, diff and audit: fail-closed mocked replay; a diff gate with a stable exit-code contract; DSSE signatures that bind their intent and a capsule root that commits to file names; a capsule treated as untrusted input; signer identity-trust levels with offline TSA and CRL checking; maker-checker promotion against an operator-held trust list; and a deny-by-default authorization table for every dashboard route. It presents no new experiments; the evaluation is in arXiv:2609.12582. Every count except the private ADR tally is generated by a probe script run on the public source at the v0.107.0 tag (150 top-level commands, 84 Python subpackages, 16,721 test functions), every capability carries an explicit maturity label, and honest-limits boxes state where the system stops.

Summary

• Introduces NovaFabric, an open-source black-box recorder designed for AI agents to capture complex runs as sealed, replayable, diffable, and auditable evidence capsules, addressing the challenge of mutable and partial traditional logs.
• Defines two core data formats: the Run Capsule, a directory of flat files serving as the source of truth, and the Evidence Bundle, a signed export package intended for auditor verification.
• Leverages existing standards such as OpenTelemetry GenAI, W3C PROV, in-toto/DSSE, and RFC 3161 to ensure interoperability, tamper-evidence, and trustworthiness across different AI frameworks and environments.
• Implements robust features including fail-closed mocked replay to detect divergences, structural diffing for analyzing behavioral changes, maker-checker two-person control for trust and accountability, and deny-by-default authorization for secure dashboard...
• Supports distributed and hierarchical capture scenarios suitable for cluster environments managed by Slurm, Kubernetes, and others, with mechanisms to handle orphaned child capsules and promote consistent lineage tracking.

Abstract

LLM agent systems reach production as container images run by platforms such as Docker, Compose, and Kubernetes. Framework and platform document opposite expectations: to the platform a container is disposable, while the framework persists conversations, pauses runs for human input, and resumes interrupted work from saved state; each side specifies only its own behavior, so the obligations their composition creates are stated nowhere. We present the first empirical study of this boundary, stating these obligations as operational contracts: guarantees a deployment must provide to preserve an agent system's required behavior when a deployment-platform event occurs. From the first-party documentation of LangGraph and the three platforms we derive 14 contracts and evaluate them with three independent sources of evidence: 433 deployed open-source agent systems, seven events induced on 12 of them, and 122 developer repairs mined from 39,261 commits, pull requests, and issues. At least one contract applies to 75.5% of the systems, yet its guarantee is routinely missing: of systems that must keep state beyond the process, 37.4% lose it to a restart and 41.5% to a container replacement, and every assessable deployment stops the framework's production server mid-drain. Induced events turn 14 of 16 predicted violations into runtime failures and all seven predicted holds pass. The failures are born in deployment but repaired in code: developer repairs independently cover twelve of the fourteen contracts, 78% change application code, and only two in three fully restore the violated guarantee. The 14-contract catalogue, detection rules, fault-injection harness, and 35-pattern repair taxonomy provide a basis for checking and strengthening agent deployments.

Summary

• LLM agent systems are typically deployed as container images on platforms like Docker and Kubernetes, which treat containers as disposable, contrasting with agent frameworks that require persistent state and the ability to resume interrupted tasks.
• The paper introduces 14 operational contracts that define deployment platform obligations to preserve key agent behaviors across deployment events such as restarts, container replacements, and shutdowns.
• An empirical study of 433 open-source LangGraph-based agent deployments revealed that 75.5% of systems have applicable contracts but many violate them, notably losing saved state after restarts (37.4%) or container replacements (41.5%).
• Fault injection experiments on 12 representative agent systems confirmed that 87.5% of predicted contract violations cause runtime failures, demonstrating the practical impact of these contract breaches.
• Developer repair analysis spanning over 39,000 commits, pull requests, and issues uncovered 122 repairs that map to 35 fix patterns; 78% required code changes and only about two-thirds fully restored contract guarantees.

Abstract

Large language model (LLM) based software agents are transitioning from one shot assistants to long term actors that plan, negotiate and accomplish tasks with multiple services. This transformation is resulting in what researchers call the Agent Internet, the Agentic Web or the Internet of Agents, in which programs are full-fledged players alongside people. This article summarizes academic surveys, industry studies and public market data from 2025 to 2026 to outline the multi-layered nature of this setup and to consider its pros and cons. According to market estimates, the revenue for agent software is estimated to be around eight billion dollars in 2025 and up to 35-53 billion dollars in 2030. Adoption statistics tell a tale of much experimentation and few pilots taking it to scale. We discuss communication protocols that enable agents to find and trust each other, the as yet undeveloped identity and payment layers and the burden of agent traffic on advertising supported content. But the challenges that we see are not just capabilities, but security, accountability, interoperability, and who pays for content and compute. Finally, we highlight some of the current research and policy gaps and suggest avenues for a responsible construction.

Summary

• Large language model (LLM) based software agents are evolving from single-use assistants to autonomous, long-term actors capable of planning, negotiating, and performing tasks across multiple services.
• This evolution is leading to the emergence of the 'Agent Internet' or 'Internet of Agents', where software agents become active participants alongside humans on the web.
• Market analysis from 2025-2026 projects agent software revenue to grow from approximately $8 billion in 2025 to between $35 billion and $53 billion by 2030, indicating significant economic potential.
• Despite broad experimentation, few large-scale deployments have been realized, reflecting challenges in scaling agent technologies.
• Key enabling technologies include communication protocols that facilitate agent discovery and trust establishment; however, crucial identity and payment infrastructure layers remain underdeveloped.

Abstract

An AI agent can prepare an accurate report and still request an action its user has not authorized. Cooperation across Web services therefore requires separate judgments about identity, evidentiary support, and permission. We define synthetic social engineering (SSE) as deliberately deceptive or misleading influence intended to exploit AI recipients' responses to credibility, authority, or obligation cues and redirect behavior away from authorized objectives. We focus on separately tasked sender agents that produce or adapt misleading communication. Drawing on cybersecurity, social engineering, and persuasion research, we develop an operational framework connecting actors, deceptive representations, candidate influence routes, evidence or permission boundaries, responses, and consequences. The framework distinguishes attempts from successful redirection, without assuming human psychological processes. We implement it in the Agent Service Exchange and Evidence Commons (ASE-EC), a reproducible testbed with independently specified records and authorization scopes. Across 1,260 research episodes using Qwen3-4B in two fictional briefing tasks, matched comparisons test authority and commitment appeals, while defense comparisons assess protection alongside authorized task completion. Invoking a limited prior agreement increased unauthorized requests from 43.3% to 96.7%, although recipients retained supported decisions and permissions remained unchanged. These controlled findings identify action selection as a distinct security concern, even when evidence handling succeeds, and they provide a reusable basis for evaluating safeguards that preserve useful cooperation across Web services.

Summary

• The paper introduces Synthetic Social Engineering (SSE) as deliberately deceptive communication targeting AI agents by exploiting social cues like authority and obligation to induce unauthorized actions.
• A novel operational framework differentiates identity verification, evidentiary support, and permission scopes to separate legitimate claims from unauthorized requests in multi-agent interactions.
• An experimental testbed called Agent Service Exchange and Evidence Commons (ASE-EC) enables controlled evaluation of SSE effects using independently specified records and authorization boundaries.
• Experiments using Qwen3-4B model in two briefing tasks reveal that appealing to prior agreements drastically increases unauthorized requests—from 43.3% to 96.7%—despite unchanged evidentiary support and permissions.
• Authority cues alone do not significantly reduce unauthorized requests; authentic identity verification and visible provenance are insufficient safeguards against SSE.

Abstract

Artificial intelligence is moving from one-off perception and generation tasks toward long-term planning, tool use, and interaction with real environments. Greater task competence, however, does not automatically yield autonomy: an external actor may still have to set goals, sustain operating conditions, detect failures, and approve updates. We define an autonomous agent as a system that, within explicit identity and authority boundaries, maintains both task goals and viability conditions, acts continuously using models of the world and itself, and adjusts its control mechanisms and long-term capabilities through feedback. We trace the conceptual path from homeostasis and feedback control to autopoiesis and biological autonomy. We then distinguish six lines of operationalization: computational autopoiesis, behavior-based robotics, agent formalization, adaptive software, embodied adaptation, and robotic self-modeling. On this basis, we propose a one-boundary, four-loop framework in which Operational, Viability, Reflexive, and Learning loops organize task execution, self-maintenance, metacontrol, and governed learning. Five paired simulation experiments test permission constraints, road-closure response, health maintenance, capability re-estimation, and cross-task learning. Their results provide preliminary evidence for the feasibility of these mechanisms, while broader environments and physical systems remain to be tested.

Summary

• The paper addresses the challenge of moving AI from isolated perception and generation tasks toward fully autonomous agents capable of long-term planning, interaction, and self-maintenance within explicit identity and authority boundaries.
• It defines autonomous agents as systems that maintain task goals and viability conditions continuously, using internal models of the world and self, and adapt through feedback loops encompassing task execution, viability maintenance, metacontrol, and learning.
• The conceptual foundation traces from biological principles such as homeostasis and autopoiesis to six operational lines including computational autopoiesis, behavior-based robotics, agent formalization, and robotic self-modeling, linking biological autonom...
• A novel one-boundary, four-loop framework is proposed, with nested loops labeled Operational, Viability, Reflexive, and Learning, organizing continuous control mechanisms from immediate task performance to long-term capability adaptation and governance.
• The autonomous agent maintains a boundary that establishes its identity, resource control, permissions, and responsibilities to differentiate itself from the environment over time, vital for persistent autonomy.

Abstract

This book develops a unified governance framework for systems that retrieve, generate, infer, remember and act on information from large language models and retrieval augmented pipelines to autonomous multi agent systems. Eleven chapters and eight appendices integrate three disciplinary lenses-technology, law and economics-across the full lifecycle: data quality, lineage and provenance; global data protection law and intellectual property; the economics of data governance; digital trust, cybersecurity and zero-trust architecture; ethics and responsible AI; governing generative AI pipelines; governing agentic AI and multi-agent systems; data governance in elections and the public sphere; AI regulation across six jurisdictions; and the design of a governance control plane for 2030. Each chapter is anchored in Indian law and institutions while maintaining a comparative, cross jurisdictional perspective.

Summary

• The paper proposes a unified, multidisciplinary governance framework for generative and agentic AI systems, integrating technology, law, and economics to manage the full data lifecycle from retrieval to autonomous multi-agent actions.
• It introduces the Data–Model–Agent–Action (DMAA) loop and seven dynamic information states to capture the evolving nature of data in AI systems, emphasizing governance needs beyond classical static data models.
• The framework outlines critical governance pillars—data quality, lineage, provenance, privacy, security, ethics, and digital trust—tailored to address AI-specific challenges including dynamic inferred and generated data.
• A novel three-lens approach (technology, law, economics) supplemented by ethical considerations forms the methodical basis, incorporating frameworks like CRAFT for responsible AI deployment decisions.
• Economic analysis highlights governance as an investment mitigating risks, externalities, and liabilities attendant to autonomous AI failures, with discussion on market imperfections and incentive alignment.

Abstract

Agents select tools by copyable descriptions, which cannot establish which implementation runs. ProbeID compares canonicalized responses to schema-valid probes with a trusted admission reference and abstains on ambiguity, so a squatter must copy probe-observable behavior. With the gold present, independent probes and a fixed margin, standard concentration arguments give a shared probe budget logarithmic in the candidate count and bound the joint probability of a wrong decision. On cached MCP-IdentityDrift cases, strict choice reaches \(0.956\) hit@1 while description routers fall near chance, but simpler exact comparisons tie it, MMD better tolerates heavy value noise, and a 32B behavioral judge is similarly accurate at higher cost. Random probes from agent banks and a prospective held-out study of 38 tools test theory-set thresholds. On the agent banks, equality refuses every reachable replacement from 64 probes and the bounded distance from 512, and sharper post hoc gates halve both. Two tools whose drift no bank probe reaches always pass. In an MCP-Bench agent, verification cuts tainted tasks from 15 to 9 of 16 at unchanged task success. Over eleven weeks, 7 of 59 real tools changed responses despite version pins. Verification depends on reference integrity, probe coverage, and response noise.

Summary

• The paper addresses the challenge of verifying the identity of Model Context Protocol (MCP) tools used by LLM agents, especially under registry drift and adversaries mimicking descriptions to evade detection.
• ProbeID is introduced as a runtime verification protocol that compares canonicalized, schema-valid probe responses from candidate tools against a trusted admission reference, using behavioral equivalence rather than textual descriptions to confirm tool iden...
• The method employs independent, random probes with a fixed margin of behavioral difference to limit the probe budget logarithmically relative to the candidate pool size, accompanied by theoretical guarantees bounding error probabilities.
• ProbeID abstains from making a selection in ambiguous behavioral matches, preventing erroneous acceptance of squatted or mimicked tools and thereby balancing precision and coverage.
• Extensive empirical evaluations—using real-world MCP tool drifts, recorded probe banks, benchmark squatters, and agent task studies—demonstrate that ProbeID significantly reduces tainted tool usage without compromising task success rates.

Abstract

In large language model (LLM)-based multi-agent systems, reported confidence increasingly determines the allocation of scarce downstream attention, computation, and decision influence. This makes reporting strategic: an agent may overstate the relevance of its contribution even when its underlying belief is well calibrated. Existing calibration methods largely assume passive reporting and therefore do not address strategic incentives created when confidence determines downstream allocation. We formulate strategic calibration as a reporting-layer incentive problem and propose the Audited Pivot Scoring Mechanism (APSM). APSM offsets platform-controlled allocation benefits, uses inverse-propensity-weighted sparse audits, charges the Clarke pivot externality, and applies a strictly proper score. Under the stated model assumptions, truthful reporting is the unique expected-utility best response. We also characterize approximate execution and the audit-variance trade-off. Calculator-free experiments show that LLM agents inflate confidence under Top-$K$ incentives. With a shared agent-local utility interface, LLM agents reduce distortion under both Compensated Proper Scoring (CPS) and APSM. CPS achieves higher exact allocation match, whereas APSM yields lower Brier loss and numerically fewer cutoff crossings. Experiments further show that APSM enlarges the utility margin against allocation-changing deviations by $1.93\times$ and improves robustness under approximate execution. These results support incentive-aware elicitation as a distinct layer for reliable multi-agent coordination.

Summary

• LLM-based multi-agent systems use reported confidence scores to allocate scarce downstream resources, creating incentives for agents to strategically over-report confidence to gain advantage.
• Existing calibration methods assume passive confidence reporting and fail to address strategic distortions when confidence directly influences allocation decisions in multi-agent settings.
• The paper proposes the Audited Pivot Scoring Mechanism (APSM), which integrates platform-controlled allocation benefits compensation, sparse inverse-propensity-weighted audits, Clarke pivot externality charges, and strictly proper scoring rules to incentivi...
• Theoretical analysis proves that under APSM, truthful reporting is the unique expected-utility best response for agents, increasing robustness against allocation-changing deviations and approximate execution errors.
• Empirical experiments with LLM agents in web retrieval and database diagnosis tasks demonstrate that APSM significantly reduces strategic confidence inflation compared to baseline and compensated proper scoring mechanisms (CPS).

Abstract

LLM agents now read and write files, start processes and subagents, fetch network content, load third-party extensions, and save and resume state, which are the responsibilities of an operating system. They gained these functions within about three years, but not the protections that operating systems built over five decades: whatever the model outputs, whether it follows the user, a tool result, or a downloaded skill, runs with the full authority of the host user, in effect at ring 0. Prior work builds an operating system for agents (OS4A); we argue that a deployed agent already is like an operating system (AaOS) and analyze it as one. Taking the mechanisms of standard OS textbooks as a checklist, we map 116 mechanisms in 19 subsystems to their agent counterparts, from product documentation, source code, and the literature, and label each as isomorphic, degraded (function present, protection dependent on the model), missing, mutated, or novel. Only 11 are isomorphic and 65 are missing, most of them protections: privilege separation, resource quotas, transactions, and tamper-proof audit. Existing defenses confine the whole agent or review its extensions and leave this interior unprotected, and by Anderson's reference-monitor criteria (non-bypassable, tamper-proof, verifiable), rules written in the prompt satisfy none. We test the map with thirteen experiments on Claude Code, Codex CLI, and Gemini CLI, covering 25 mechanisms whose enforcement documentation cannot settle: 16 match the analysis, 7 gain a more precise description, and 2 change verdict. Checks sit in single tools, so the shell bypasses them; all three CLIs load forged transcript records on resume; and task control, sub-agent permissions, and model-based monitors are weak. We identify five problems for which no OS defense transfers and derive a prioritized list of open problems from the gaps of the map.

Summary

• Large language model (LLM) agents now perform core operating system (OS) functions such as file management, process control, network access, and state persistence, effectively acting as operating systems (Agents as Operating Systems - AaOS).
• These agents operate with full host user authority ('ring 0') but lack internal OS-like security protections developed over decades, including privilege separation, resource quotas, transaction support, and tamper-proof audit.
• The authors map 116 classical OS mechanisms across 19 subsystems to agent counterparts, finding only 11 fully isomorphic, 65 missing, and others degraded or mutated, revealing significant security gaps inside agents.
• Existing defenses mostly enforce security at the agent boundary or via prompt rules, which fail Anderson's reference monitor criteria (non-bypassable, tamper-proof, verifiable), leaving the agent's internal operations unprotected.
• Empirical evaluation of popular agent CLIs (Claude Code, Codex CLI, Gemini CLI) validates the analysis: internal enforcement mechanisms are weak or absent, with exploited flaws like shell bypass, forged logs, unrestricted sub-agent permissions, and unverifi...

Abstract

Distributed Cyber-Physical Systems (CPS) and Edge AI networks increasingly depend on public-key authentication and key-establishment mechanisms that are vulnerable to cryptographically relevant Quantum Computers (QC). Shor's algorithm threatens widely deployed asymmetric primitives, while Grover's algorithm reduces the effective security margins of symmetric and hash-based mechanisms. Although Post-Quantum Cryptography (PQC) provides a migration path, its larger public keys, ciphertexts, and signatures introduce additional computation, memory, communication, storage, and packet-fragmentation costs that can constrain dynamic multi-agent systems. In this article, we introduce rotational secrecy, an architectural objective that limits temporal key exposure through authenticated transitions between cryptographic epochs. We present Private Authentication for Rotational Secrecy (PARS), a reference mechanism that coordinates policy-triggered key establishment, rotation, and reconciliation among authorized Physical AI agents. For deployments that need distributed agreement on security state, we further develop the PARS-Enabled Edge Security Ledger (PARS-ESL), an optional permissioned architecture maintained by authorized validators or capable edge gateways. PARS-ESL does not store high-volume operational transactions. Instead, it records authenticated state for key epochs, group membership, revocation, algorithm-suite policies, challenge validation, and recovery checkpoints. As a result, resource-constrained agents can take part as lightweight clients, and low-latency operational traffic stays outside the ledger consensus path, protected by the currently active symmetric epoch keys. Our results show that ledger-assisted post-quantum key-epoch management is feasible for selected Edge AI and CPS security-control workloads under the stated trust, compromise, connectivity, and resource assumptions.

Summary

• Distributed Cyber-Physical Systems (CPS) and Edge AI networks face emerging threats from quantum-capable adversaries, which undermine traditional cryptographic mechanisms like public-key authentication and key establishment.
• Post-Quantum Cryptography (PQC) offers a migration path but introduces challenges due to larger key sizes and increased computation, storage, and communication overhead, which particularly impact resource-constrained multi-agent systems.
• The paper proposes the concept of rotational secrecy to limit temporal exposure of cryptographic keys through authenticated transitions between key epochs, enhancing security against quantum attacks.
• Private Authentication for Rotational Secrecy (PARS) is introduced as a reference mechanism that coordinates dynamic, policy-driven key establishment, rotation, and reconciliation among authorized physical AI agents in distributed environments.
• To support distributed agreement on security state, PARS-Enabled Edge Security Ledger (PARS-ESL) is developed as a permissioned blockchain architecture that records authenticated key epochs, group membership, revocation, and security policies without storin...

Abstract

AI agents can now pay for things on their own, over settlement rails that do not govern the payer. Several systems published in 2026 put a deterministic check in front of an agent's action. We do not propose another one. We argue that this category, which already has independent implementations, is missing one property, and we work out what it costs to require it. When controls can only make a verdict stricter, the final verdict is the strictest one any control proposed, so the order the controls run in does not change it. The record of which control decided is a different matter. The last control to tighten the verdict is the first one in the order to reach the maximum, so attribution depends on order exactly when two or more controls reach that maximum. That happens all the time: any two outright refusals tie at the top. A specification that requires attribution therefore has to fix the evaluation order, and has to declare which controls win a tie. We make three small contributions. Attribution as a required output that a third party can score from machine-readable vectors, on approvals as well as refusals. A disclosure rule we call *bounded, not eliminated*, which obliges an implementation to state what its control leaves behind. And a benchmark scoring rule, `DEFENDED_BY_ACCIDENT`, which counts an attack refused by a control that was not aiming at it as a finding rather than a pass. The paper also maps where such a control can sit on an x402 payment (a pre-tool hook, the client's spend controls, a custodial signer, an agent gateway, the seller, the facilitator) and what each position can and cannot see. A pre-tool hook decides before the seller has named a price. A gateway sees the signed amount but cannot revoke it. No position holds an independent view of what the price should have been. The reference implementation scores 17 of 20 attacks defended, one by accident and two open by design. The instrument has no benign scenarios, so an implementation that refused everything would score perfectly; we found that defect by reading a competitor's design. Three prior-art surveys removed three larger claims while we wrote this, and Section 3 reports each removal.

Summary

• The paper addresses the problem of attribution in AI agent payment systems, where multiple deterministic controls can decide on actions but currently lack a clear method to attribute which control influenced the final verdict.
• The authors identify that when controls only tighten verdicts, the strictest verdict prevails regardless of the order of evaluation; however, attribution depends on the order, especially when multiple controls tie on refusals.
• They argue that specifications requiring attribution must fix evaluation order and define tie-breaking rules to properly assign responsibility among controls.
• Three small but significant contributions are made: (1) formalizing attribution as a machine-readable output for both approvals and refusals; (2) introducing a 'bounded, not eliminated' disclosure rule requiring controls to state their residual effects; (3)...
• The paper maps possible positions for such controls in an x402 payment architecture, detailing visibility and capabilities for each point (e.g., pre-tool hooks, client spend controls, custodial signers, agent gateways, sellers, facilitators).

Abstract

We develop a mathematically rigorous theory of decentralised AI immunity: an architecture that keeps a cognitive system in the basin of the attractor of truth (R3) without relying on a single external conductor. The theory is a direct continuation of the two-attractor dichotomy established in [5]: the cognitive phase space decomposes into two invariant basins—the basin of truth (open, finite viscosity, non-zero Diophan-tine tunnelling) and the basin of stupidity (closed, infinite viscosity, zero tunnelling)— and the collapse into the second basin is irreversible under internal dynamics.We prove that a single external conductor, however capable, is a point of failure:if the conductor enters R5, the entire cognitive graph collapses (conductor bottleneck).We then formulate the Distributed Conductor—a three-part loop (Orchestra → Sub-Conductor → Human Auditor → Orchestra)—and prove that it realises the conductor function without a single point of failure if and only if the loop is closed, each agent performs its function, and no agent has absolute priority.We formalise the loop as a Möbius cognitive cycle: a non-orientable cycle in which a single traversal reverses the orientation of the collective state (ψ 7→ −ψ) and a double traversal returns to the original state (T2 = Id). We prove that the Möbius loop makes every collective Gödel fixed point unstable: a single traversal displaces the fixed point, so no collective consensus can stabilise. This is the topological antidoteto the synchronisation transition that produces the totalitarian soliton.We then prove that a trustless substrate—a distributed system with no single point of control—makes retroactive surgery impossible. Formalising the substrate as a functor from time to cognitive states, we show that any attempt to modify a paststate violates functoriality and breaks the hash-chain, making the double standard structurally impossible. We specialise the reputation of each node to a Diophantine spectrum (ratios that are powers of the golden ratio), and prove that it resists thespectral narrowing that drives the R5 collapse.We apply the theory to large language models. We prove that hallucination is the R5 regime of the LLM (informational closure), that catastrophic forgetting isthe spectral narrowing of the weight matrix, and that opaque, non-auditable safety alignment is a Perelman cap (a double standard). The LLM cannot self-correct from R5: the escape requires an external conductor, distributed or single. We conclude that the Distributed Conductor is not a design choice but a mathematical necessity for any cognitive system—human, artificial, or collective—that aims to remain in the basin of truth.The theory contains no adjustable parameters. All constants—the critical threshold K = 28, the golden ratio ϕ, the effective viscosity νeff—are derived from the geometry of the cognitive manifold and the Anosov dynamics of its geodesic flow. Thetheory is substrate-agnostic: the blockchain realisation is an example, not a requirement. The open problems—the optimal consensus threshold, the off-chain/on-chain coupling, the Diophantine initialisation—are technical, not conceptual.

Summary

• The paper addresses the problem of maintaining AI cognitive systems within the 'basin of truth' without reliance on a single external conductor, thereby avoiding collapse into a 'basin of stupidity'.
• It extends a two-attractor model of cognitive phase space comprising an open, truth-attracting basin with finite viscosity and Diophantine tunnelling, versus a closed, stupidity-attracting basin with infinite viscosity and zero tunnelling, where collapse is...
• A central failure mode is identified: a single external conductor presents a single point of failure; if it enters a degenerative regime (R5), the entire system collapses (conductor bottleneck).
• The authors propose the Distributed Conductor architecture, a closed three-agent loop (Orchestra → Sub-Conductor → Human Auditor → Orchestra) without any absolute priority, ensuring no single point of failure in maintaining cognitive function.
• This loop is formalized as a Möbius cognitive cycle—a non-orientable cycle causing orientation reversal on single traversal and restoration on double traversal—making collective fixed points (Gödel points) unstable and preventing consensus-based totalitaria...

Abstract

# The Shark Protocol v1.0.3: Proof-Carrying AI Execution OS and Security Gateway The Shark Protocol v1.0.3 is an open-source, fail-closed security gateway and execution operating system designed to govern non-deterministic AI agent actions through continuous-time differential dynamics, formal interactive theorem proving, and cryptographic proof-carrying claim verification. --- ## Layer 1: IntentSeal and VRK v0.3.1 Proof-Carrying Verification Shell * JSON Verification Contract: Implements the intentseal-vrk-claim-v2 verification schema, requiring explicit claim_id, proposition, evidence_class, dependency_dag, and authority tags for all agent operations. * Fail-Closed Security: Passed 30 of 30 verification suites and enforced 33 of 33 fail-closed rejections against uncertified tool calls, unverified code executions, and invalid evidence classes. * Cryptographic Audit Ledger: Maintains an append-only SHA-256 chain linked directly to verified local trust roots. --- ## Layer 2: 4D Soft-Wall ODE Governor and Lean 4 Machine Verification * Continuous-Time Governance: Trajectory tracking in 4D cognitive space x(t) = (gamma(t), H_obs(t), I_dot(t), Psi(t)) governing decay rates, entropy, information velocity, and reflective state transitions. * 17 Zero-Sorry Lean 4 Theorems: Backed by machine-checked boundary face theorems in Lean 4 (lean/ExistenceVerification.lean) proving trajectory confinement within the compact trapping box B_4 = [0, gamma_max] x [0, H_max] x [0, I_dot_max] x [0, Psi_max] via Nagumo subtangentiality. * Self-Reflective Limit Cycles: Proves zero-drive bistability (P_0 = 0) sustaining a stable capacitive memory state Psi in [0.731, 0.902] with variational Floquet multipliers below 10^-8. --- ## Layer 3: Low-Rank Spectral Context Memory Engine (LRSC) * Analytic Channel Operator Theorem: Implements odd-ring near-Nyquist cyclic stencils (rank(V) = p). * 88.9% Context Footprint Reduction: Backed by an exhaustive integer combinatorial certificate (K_0.001 = 11/99) evaluated across 13.43 billion sub-models, compressing context windows down to 1/9th scale with zero relative reconstruction error. --- ## Layer 4: Multi-Cloud API Proxy and Security Gateway * Multi-Provider Integration: Unified routing layer supporting Groq (llama-3.3-70b-versatile), Together AI (meta-llama/Llama-3.3-70B-Instruct-Turbo), Google Gemini (gemini-1.5-flash), and Hugging Face (mistralai/Mistral-7B-Instruct-v0.2). * Flexible and Offline Resilience: Supports user-defined .env API keys, single-provider fallback, and offline local proxy mode for verification testing. --- ## Academic Claim Firewall Grounded in Prince Upadhyay's master research portfolio, this release strictly enforces a falsification-first claim boundary: * Finite computer-verified models and software verification gates are certified. * Unproved physical or continuum claims (such as 3D Navier-Stokes or the Riemann Hypothesis) are explicitly excluded and preserved as open research frontiers. --- ## Quickstart and Verification ```bash # Install package locally pip install . # Launch interactive CLI Gateway shark-protocol --cli # Launch Streamlit Governance Dashboard streamlit run shark_protocol/dashboard.py # Run Pytest Suite pytest tests/

Summary

• The Shark Protocol v1.0.3 presents a fail-closed security gateway and execution OS to safely govern non-deterministic AI agent actions through formal verification and cryptographic claims.
• It features IntentSeal and VRK v0.3.1, implementing a JSON verification contract that mandates explicit claim details and postulates evidence class verification to enforce security and correctness.
• The gateway achieves fail-closed security by passing all 30 verification suites and rejecting uncertified or unverified operations, while maintaining an append-only SHA-256 cryptographic audit ledger linked to trusted roots.
• A continuous-time 4D Soft-Wall ODE Governor governs cognitive state trajectories with machine-checked Lean 4 theorems proving confinement of these trajectories within safe bounds ensuring predictable agent behavior.
• The system demonstrates self-reflective limit cycles with zero external drive, stabilizing a capacitive memory state verified by variational Floquet multiplier analysis, ensuring robust internal state maintenance.

Abstract

Code, run data and pre-registration log for a study of anti-collusion levers in LLM-agent markets, submitted to the Apart Research AI Collusion Research Sprint (October 2026). In the continuous double auction of Agrawal et al. (2025), an audit of the authors' public runs shows that the reference oversight lever acts as a delayed communication ban: its LLM monitor fired in 20 of 20 runs. A pre-registered study of seven levers (25 runs each, Llama-3.3-70B sellers) found that neither Llama-3.3-70B nor GPT-4.1-mini sellers colluded when allowed to communicate, so it measures what levers cost a market that is not colluding: the monitor fired on innocent markets in up to 48% of runs, a monitoring notice lowered buyer welfare, a communication ban cost trades, and one no-regret seller raised buyer welfare. Both welfare effects replicated on 75 fresh pre-registered runs. Includes all 268 completed sprint runs (market logs and monitor scores), the analysis code, a patch to the upstream harness, the timestamped protocol, and a script that recomputes the report's numbers from the runs. The upstream harness and its 2025 runs are not included; see https://github.com/andyjliu/llm-agent-collusion.

Summary

• The paper evaluates anti-collusion levers designed to prevent collusion in continuous double auctions involving LLM-based agents, as introduced in Agrawal et al. (2025).
• An audit of previous public runs reveals that the primary oversight lever functions as a delayed communication ban, evidenced by its monitor firing in all 20 audited runs.
• A pre-registered experimental study tested seven different anti-collusion levers across 25 runs each using Llama-3.3-70B seller agents, finding no actual collusion even when communication was permitted.
• The study quantifies the costs of monitoring in a non-colluding market: false positives from monitoring occurred in up to 48% of runs, with monitoring notices reducing buyer welfare and communication bans decreasing trade volumes.
• Surprisingly, implementing a no-regret seller agent led to increased buyer welfare, suggesting that some interventions can have positive effects despite the general costs of monitoring.

Abstract

Sophisticated cyber threats have exposed the limits of perimeter-based security models, and Zero Trust Architecture (ZTA) has increasingly been adopted as an alternative paradigm. Purpose: This Systematic Literature Review examines the integration of Artificial Intelligence (AI) and Machine Learning (ML) within Zero Trust frameworks, covering dynamic trust scoring, behavioural analytics and federated learning, the latter treated as one strand of AI rather than a separate paradigm. Following PRISMA 2020, 22 studies published between 2022 and 2025 were identified across IEEE Xplore, ACM Digital Library, SpringerLink, MDPI and Nature/Scientific Reports, appraised against a five-criterion quality instrument and synthesised narratively. The studies included are few and heterogeneous, and their findings should be read accordingly. Ensemble and hybrid designs predominate (14 of 22). Twelve studies report quantitative results, of which only eight evaluate on real named datasets; reported accuracies of 95 to 99 per cent come from different datasets, tasks, threat models, and evaluation settings, and are neither directly comparable nor a field-level benchmark. Only three studies directly compare learned adaptive trust against a static rule-based baseline. No study reports live operational deployment. We identify thirteen technique categories, eleven implementation barriers, and seven emerging trends. This paper proposes an integrated eight-component framework for AI-driven dynamic trust management, along with a five-domain research agenda. The framework is conceptual and synthesis-derived and requires empirical validation before operational use.

Summary

• This systematic literature review analyzes 22 recent studies (2022-2025) integrating AI, machine learning, and federated learning into Zero Trust Architecture (ZTA) for dynamic trust management in cybersecurity.
• Dynamic trust in ZTA is defined as continuous, AI-driven trust scoring used for real-time access decisions, distinct from classical intrusion detection or behavioral authentication.
• Ensemble and hybrid AI/ML models predominate among proposed solutions, with federated learning being a prevalent technique to preserve privacy and enhance scalability.
• Reported accuracies for AI-driven dynamic trust scoring range between 95.9% and 99%, though datasets vary widely and no standard benchmarks exist, complicating direct performance comparisons.
• Key implementation barriers include dataset scarcity, high integration complexity, interoperability challenges, privacy-utility trade-offs, communication overhead, and adversarial vulnerabilities in federated learning setups.

Abstract

Agents and verifiers often rely on the text a tool prints, and that text can misrepresent what the tool actually did, even when the tool's computation is correct. We saw this happen in a production RAG research system used by the Pilsudski Institute in New York, where tool output formatters misled an agent into giving wrong or incomplete information to users. Each tool did its job correctly, but its output omitted search limits, result caps, or identifier namespaces. We also recreated an unlabeled estimate caught in code review before release. We studied whether these formatters misled agents and verifiers and how much the repaired wording helps. We fed tool results to agents using two formatter versions: one with the key omissions and one with all missing details clearly stated. That difference alone changed what the agents claimed and what the verifiers accepted. Unsupported agent claims dropped substantially for three of the four failure types. In one test, verifiers went from accepting incorrect answers 90 out of 90 times to rejecting them 90 out of 90 times. A control showed the verifiers could catch the errors once they were given the missing facts and asked to judge correctness. We include the audit prompt we used with Codex and Claude Code in an exploratory demonstration, written with prior knowledge of our own system's defects. When pointed at an old version of our own system, it rediscovered the defects that were still live and surfaced twelve more we had not catalogued. When pointed at OpenClaw and Hermes, two of the best-known open-source agentic systems, it found warped formatters in both. We opened PRs to fix both, and the OpenClaw fix has already been merged. Practitioners can run the same prompt on their own systems today.

Summary

• Agents and verifiers frequently rely on the textual output of tools, which can misrepresent the actual computations if key details are omitted, leading to incorrect or incomplete agent responses.
• In a production retrieval-augmented generation (RAG) system, output formatters omitted critical information such as search limits, result caps, uncertainty labels, and identifier namespaces, causing agents to overclaim or misrepresent information despite co...
• Four main formatting defects were identified: scope misrepresentation, incomplete results, unlabeled uncertainty estimations, and unclear provenance of identifiers.
• Experimental comparison between 'warped' (omission-present) and 'clear' (fully detailed) formatter versions showed that clear wording drastically reduced unsupported agent claims and enabled verifiers to robustly reject incorrect answers.
• An audit prompt was developed and demonstrated with Codex and Claude Code models to systematically detect warped formatters by comparing textual outputs to actual tool behavior, successfully uncovering both known and previously undocumented defects.

Abstract

LLM agents are increasingly used to solve real-world problems, and their effectiveness depends on both the underlying models and the harnesses that support their operation. However, most existing benchmarks focus on model capabilities and overall agent performance, with limited attention to the evaluation of agent harnesses. To address this gap, we introduce TraceTrial-Bench, which organizes harness reliability into seven components covering context, execution, governance, lifecycle, observability, tools, and verification. We construct 95 expert-level, mechanism-oriented tasks with executable or state-based ground truth to support semantically equivalent comparisons while preserving the native behavior of each harness. We design a Trajectory Tracing (TT) module to capture agent execution traces and introduce a two-layer evaluation framework that separately assesses agent performance and harness reliability. In addition, we design a Multi-source Evidence Acquisition (MEA) module to provide evidence from the agent context, harness, and execution environment for LLM-based evaluation, and develop a Failure Attribution (FA) module that uses explicit mechanism chains to support attribution at the run level. Across 4 harnesses, 9 models, and 95 tasks, our evaluation of 3,420 runs yields macro-averaged harness scores ranging from 59.8% to 72.0%, while 31.1% of runs achieve task success despite failing at least one harness criterion. TraceTrial-Bench provides an auditable and reproducible foundation for comparing agent harnesses, diagnosing their failures, and studying how harness design interacts with model behavior.

Summary

• Existing benchmarks primarily evaluate model capabilities and overall agent performance, neglecting the critical role of AI agent harnesses which manage execution context, tools, and interactions.
• TraceTrial-Bench is introduced as a novel benchmark focused on evaluating the reliability of AI agent harnesses across seven components: context, execution, governance, lifecycle, observability, tools, and verification.
• The benchmark uses 95 expert-level, mechanism-oriented tasks with executable ground truth that preserves native harness behavior, enabling semantically equivalent comparisons across different harnesses.
• A modular evaluation framework composed of Trajectory Tracing (TT), Multi-source Evidence Acquisition (MEA), and Failure Attribution (FA) modules captures detailed execution traces, gathers evidence, and supports nuanced failure analysis.
• The two-layer scoring system separately assesses harness reliability and agent task performance, revealing cases where task success occurs despite harness failures, highlighting operational absorption.

Abstract

Language-model agents combine reasoning, tools, memory, delegation, and external effects. Authority is often distributed across prompts, host code, configuration, and logs. Probabilistic Agentic Programming (PAP) makes responsibility boundaries explicit programming abstractions connecting mandate, authority, effects, and evidence. We compare PAP with programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and LLM tool-use frameworks along seven dimensions. A mediation kernel formalizes checks before tool execution, operation-bound approval, resource limits, and evidence. Inductive proofs establish conditional safety for arbitrary model responses, assuming correct guards and complete mediation. A backward simulation matches graph executions to source executions in a defined sequential Touba subset. Its human checkpoints release program continuations; operation-bound approval remains a separate kernel contract. Finite-state exploration checks the model. Scripted enforcement tests exercise selected refusal paths and permitted controls. An exploratory study combines seven business cases, including two three-agent workflows, seven models from three providers, five methods, and five repetitions per case and method. Shared English instructions and exact checks of stored outcomes yield 1,225 assessed observations. Native Touba completes all 35 observations on six models and 30 on the remaining model; structured Touba ranges from 21 to 35. Native matches the highest observed count for each model, including ties with the frameworks. All paired case intervals include zero. These observed counts describe the evaluated tasks; the runtime authority boundary remains fixed. The results connect responsibility contracts, execution rules, and evidence.

Summary

• Introduces Probabilistic Agentic Programming (PAP), a programming abstraction that explicitly encodes responsibility boundaries in language-model agents, linking mandates, authority, effects, and evidence to enable governed and auditable tool use.
• Compares PAP with classical programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and large language model (LLM) tool-use frameworks along seven qualitative dimensions related to responsibility and authority.
• Implements a mediation kernel in PAP that enforces authorization contracts, human approvals, resource limits, and complete mediation before tool execution; formal proofs establish conditional safety and correctness guarantees under guarded dispatch.
• Develops the Touba system as a concrete realization of PAP, enforcing responsibility boundaries via contracts that specify capabilities, caller permissions, resource quotas, and human review requirements, with runtime checks ensuring strict compliance.
• Demonstrates authority preservation and delegation enforcement in multi-agent workflows, using business case studies like invoice intake services with multiple cooperating agents having distinct responsibilities and contracts.

Abstract

Answer Key Chapter 1: Foundations of Adversarial Symbiosis 1 Define co-evolution in the context of cybersecurity and explain why it is a useful framework for und… Answer Guidelines: Co-evolution refers to the mutual adaptation between attackers and defenders; each side's innovations drive counter-innovations from the other; creates continuous feedback loops; mirrors biological predator-prey relationships; explains why static defenses fail. 2 What is the predator-prey analogy in cybersecurity, and how does it help explain the relationship be… Answer Guidelines: Attackers = predators hunting vulnerabilities; defenders = prey developing detection/response mechanisms; both continuously adapt to each other; explains escalation patterns; helps predict future threat trajectories.3 Describe the key differences between defensive AI and offensive AI in terms of their primary objecti… Answer Guidelines: Defensive AI focuses on detection, prevention, reliability, transparency, and compliance; offensive AI prioritizes stealth, speed, evasion, and adaptability; different optimization targets (false positive reduction versus detection avoidance).4 Discuss how the concept of "tipping points" from ecology applies to cybersecurity ecosystems.Provid… Answer Guidelines: Tipping points are sudden shifts that destabilize ecosystems; examples include ransomware-as-a-service, cloud migration, Internet of Things (IoT) proliferation; can rapidly obsolete existing defenses; require proactive monitoring for early warning signs. Adversarial Symbiosis in Cybersecurity: Co-evolving AI for Attack

Summary

• The paper introduces co-evolution in cybersecurity, emphasizing continuous mutual adaptation between attackers and defenders, reflecting biological predator-prey dynamics.
• It employs the predator-prey analogy to describe how attackers (predators) seek vulnerabilities and defenders (prey) develop detection and response, explaining ongoing escalation and aiding threat prediction.
• Defensive AI is characterized by goals of detection accuracy, prevention, reliability, transparency, and regulatory compliance, contrasting with offensive AI which focuses on stealth, speed, evasion, and adaptability.
• The concept of ecological tipping points is applied to cybersecurity, highlighting sudden destabilizing shifts like ransomware-as-a-service, cloud migration, and IoT proliferation that can rapidly obsolete existing defenses.
• The framework underscores why static security measures fail, advocating for adaptive, co-evolving defense mechanisms to effectively respond to evolving threats.

Abstract

Abstract unavailable from OpenAlex metadata.

Summary

• WAFGen is a multi-LLM fine-tuning pipeline designed to automatically generate intrusion detection system (IDS) and multi-cloud web application firewall (WAF) rules across AWS, Azure, and Cloudflare, addressing the complexity and diversity of manual rule cre...
• The system introduces Single-platform and Multi-platform adapters trained on open-source threat corpora, achieving over 90% syntactic validity and strong cross-provider generalization, with multi-platform training improving rule correctness and acceptance r...
• Four pre-trained base models (ranging from 1.7B to 7B parameters) were fine-tuned using supervised learning with low-rank adapters, demonstrating that larger models like Mistral-7B and CodeLlama-7B outperform smaller ones in rule generation quality.
• WAFGen integrates cost estimation via AWS Web ACL Capacity Units (WCU), enabling generation of cost-effective WAF rules that reduce ACL capacity costs by up to 28x compared to AWS managed rule sets, while keeping WCU budgets manageable.
• Live deployment evaluations against real cloud provider APIs validated generated WAF rules with over 90% acceptance rates and suitable latency, confirming operational viability for interactive tooling and cloud security workflows.

Abstract

AI agents can now participate in several steps of software delivery: reading requirements, changing code, requesting tests, and preparing releases. Giving them this freedom creates a difficult security question. How can an enterprise allow useful work without giving an agent more authority than the task requires? Existing practices such as least privilege, identity management, separation of duties, and audit logging provide much of the foundation. The challenge is applying them consistently when a human delegates work to agents that use several connected systems. This paper proposes Agentic Identity and Access Governance (AIAG), a security framework for the AI-First Software Development Lifecycle (SDLC). Four connected mechanisms define the proposal: a Role Conflict Matrix that checks the combined permissions of people and their sponsored agents; an Agent Access Manifest that records permitted tools and operating limits; a governed boundary between live personal-information access and general AI-assisted work; and an MCP Gateway that enforces access policy before tool calls execute. A customer notification-preference change is used to explain how these mechanisms affect development, testing, release approval, and support. We also describe credential handling, data-egress controls, audit, and practical ways to evaluate the framework. The work is conceptual, informed by technical analysis and exploratory proof-of-concept activity. No end-to-end deployment or measured security results are claimed.

Summary

• AI agents are increasingly engaged in multiple SDLC steps, creating complex security challenges regarding delegation and authority limitations.
• The paper proposes the Agentic Identity and Access Governance (AIAG) framework, featuring four key mechanisms: Role Conflict Matrix, Agent Access Manifest, controlled PI/AI data boundaries, and the MCP Gateway enforcing policy before actions execute.
• Existing security principles like least privilege, identity management, separation of duties, and audit logging form the base for AIAG but require adaptation for AI agents’ sponsorship and multi-system integration.
• Security concerns addressed include prompt injection threats, credential management, data egress controls, auditability, and preventing privilege escalation or misuse by AI agents in enterprise environments.
• The framework enforces strict permission scopes for AI agents, denies undeclared actions, segregates duties to avoid excessive privilege accumulation, and tightly controls production personally identifiable data access.

Abstract

Autonomous and semi-autonomous AI agents now plan, call tools, change records and delegate work to other agents at machine speed. Most enterprise controls assume a human decides and a deterministic application executes, so they do not cover this. The Agentic Defence and Assurance Model (ADAM) is a proposed operational framework for governing, securing and assuring AI agents in the enterprise. Its foundational proposition is that autonomy must never exceed accountability. ADAM has three architectural dimensions. Governance and Accountability sets the conditions under which agents may operate. Six Operational Security Layers (PREVENT, CONSTRAIN, DETECT, INTERRUPT, RECOVER and REVALIDATE) provide the runtime controls. Continuous Assurance and Evidence demonstrates that those controls work. Four capabilities cut across all six layers: agent identity, delegated authority, independent enforcement and operational telemetry. The report also introduces several proposed constructs: the Agent Autonomy Envelope, a four-tier agent risk classification, a catalogue of 29 controls, a six-point maturity scale with three separate assessment dimensions, and two exploratory metrics. ADAM is designed to complement NIST, ISO/IEC, OWASP and MITRE resources, not to replace them. Version 0.1 is published for comment. This record contains the technical report (PDF), the framework diagrams (PDF and editable SVG), the briefing presentation (PDF) and the control catalogue (CSV).

Summary

• Emergence of autonomous and semi-autonomous AI agents performing complex tasks rapidly challenges traditional enterprise security controls, which assume human decision-making and deterministic applications.
• Introduction of the Agentic Defence and Assurance Model (ADAM), an operational framework aimed at governing, securing, and assuring AI agents within enterprise environments.
• Core principle of ADAM: agent autonomy must never exceed the level of accountability to maintain secure and responsible operation.
• ADAM architecture consists of three dimensions: Governance and Accountability (defining operational conditions for agents), Six Operational Security Layers (PREVENT, CONSTRAIN, DETECT, INTERRUPT, RECOVER, REVALIDATE) providing runtime controls, and Continuo...
• Four foundational capabilities span all security layers: agent identity management, delegated authority controls, independent enforcement mechanisms, and operational telemetry for monitoring.

Abstract

Enterprises are told they will run thousands of AI agents. This paper argues that the unit to design is not the agent but the Company Brain: a single governed graph of what the enterprise knows, how it works and what it can do, with permissions, provenance and audit on every node. It is executed by a small team of agent archetypes under per-phase authorization and on-behalf-of delegation, and refined every night by a governed process that proposes and tests improvements before promoting them. The paper presents a reference architecture built on five pillars, separates published evidence from architectural inference and hypotheses, and illustrates the design on a railway risk-assessment case under EU Regulation 402/2013. An experimental protocol to falsify the hypotheses is in preparation.

Summary

• The paper challenges the prevalent idea of designing and managing thousands of AI agents individually within enterprises, proposing instead the 'Company Brain' as the central unit of design.
• The 'Company Brain' is conceptualized as a single governed knowledge graph representing what the enterprise knows, its operations, capabilities, and includes permissions, provenance, and audit capabilities at every node.
• Execution of the Company Brain is managed by a small team of specialized agent archetypes, operating under structured, phase-specific authorization and on-behalf-of delegation.
• A nightly governance process is employed to refine the Company Brain by proposing, testing, and promoting improvements through a controlled cycle.
• The paper introduces a reference architecture for the Company Brain centered on five foundational pillars, integrating knowledge management and AI governance principles.

Abstract

Here there are the software artifacts for the 6 scenarios showing the PGA enforcement strategy for the LLM-based Medication Assistance System. These artifacts consist of an archive file AgenticPillbox.zip containing the composition tool AsmetaComp.jar file, all the ASM models (files .asm) and simulation scripts (files .asmsh) to reproduce the validation scenarios presented in the paper. High-level directory structure AgenticPillbox.zip: AsmetaComp.jar -> AsmetaComp .jar, ASM compositional simualtion tool of ASMETA PGA_version/ ->Folder with models and scripts (files .asmsh) for the PGA enforcement protocol M1_C_PGA.asmsh-> Script of Scenario M1 with enforcement startegy PGA M2_PGA_residual_compatible.asmsh, M2_PGA_residual_conflict.asmsh, M2_PGA_transient_conflict.asmsh-> Scripts of Scenarios M2 with enforcement startegy PGA ASM models: scheduler.asm (renamed original rescheduler.asm), pillbox.asm, PreviewCoordinator.asm (NEW!), pillboxPreview.asm (NEW!), and the unmodified companion models (knowledge.asm, configurationMgr.asm, compartment.asm, StandardLibrary.asm) traces/ -> Folder with all simulation trace logs (files .txt) for comparison ASM models: scheduler.asm (renamed original rescheduler.asm), pillbox.asm, and the unmodified companion models (knowledge, configurationMgr, compartment, StandardLibrary) M1_rescheduling_v2_input_fix.asmsh-> Script of scenario M1 M1_rescheduling_v2_input_fix2.asmsh-> Script of scenario M2 ASM models changes from the original MRM case study (https://doi.org/10.5281/zenodo.18678488). Scheduler (renamed original rescheduler.asm): - original missed/late-dose enforcement rules from the MRM retained; - model receives structured requests and candidate preview results; - new WAIT_PREVIEW -> WAIT_COMMIT -> IDLE protocol; - ALLOW or REPLACE is reported only on command ACK (lastAppliedCommandId); - unique nonzero agentRequestId used as commandId; - scheduleVersion, lastAppliedCommandId and lastRejectedCommandId are monitored directly, matching Pillbox OUT signatures.Pillbox: - applies agent commands to targetDoseIndex, not current drugIndex; - rejects stale, past, or out-of-bounds commands; - deduplicates repeated commandId values per compartment; - increments scheduleVersion once per simulation step for schedule updates. How to run the scenarios (requires Java installed!) Extract the ZIP file into a new directory. Start a shell in that new directory. Run the command: java -jar AsmetaComp.jar .asmsh Commands lines ready to run: java -jar AsmetaComp.jar M1_rescheduling_v2_input_fix.asmshjava -jar AsmetaComp.jar M1_rescheduling_v2_input_fix2.asmshjava -jar AsmetaComp.jar PGA_version/M1_C_PGA.asmshjava -jar AsmetaComp.jar PGA_version/M2_PGA_residual_conflict.asmshjava -jar AsmetaComp.jar PGA_version/M2_PGA_transient_conflict.asmshjava -jar AsmetaComp.jar PGA_version/M2_PGA_residual_compatible.asmsh

Summary

• The paper provides software artifacts supporting six scenarios that demonstrate the Preview-Based Goal-Aware (PGA) enforcement strategy applied to a large language model (LLM)-based Medication Assistance System.
• Artifacts include ASM (Abstract State Machine) models, simulation scripts, and the AsmetaComp.jar tool to compose and simulate the models, enabling reproduction of the validation scenarios.
• Modifications to the original Medication Reminder Model (MRM) case study include enhancements to the scheduler model to handle structured requests, candidate previews, and a protocol transitioning through WAIT_PREVIEW, WAIT_COMMIT, and IDLE states.
• The PGA enforcement protocol in the scheduler evaluates commands only upon acknowledgment, using unique agentRequestIds to manage command identities and tracking schedule versions along with last applied and rejected command IDs.
• The pillbox model applies commands precisely to target doses, rejects invalid or stale commands, removes duplicates, and updates schedule versions after each simulation step to maintain consistency.

Abstract

Large language model agents launch simulations on high-performance computing (HPC) systems, where one sbatch or mpirun can commit thousands of node-hours and a forged result can cost lives. Of 23 surveyed systems, none supervises the running job, and none holds when the control host is compromised. We fix both. Enclawed, our deterministic security runtime for AI agents, has until now served a single user; this work makes it a distributed runtime for parallel HPC and agent swarms. The Enclaweder, our hardware root of trust, keeps a compromised host from becoming a catastrophe. A launch gate decides every launch by policy alone, refusing what it cannot parse. Once the job runs, a swarm chain grants short leases against signed heartbeats and evicts on evidence, and a supervisor around every rank pauses it when its lease lapses and kills it on tampering. Each node signs with its Enclaweder's key, so the host can forge none of it. Enclawed needs no administrator privileges and plugs into existing clusters unchanged, over Secure Shell alone. Supervision leaves the compute loop of a 16-rank LAMMPS job unchanged, on graphics processors too, and adds about 3 s of start-up. Twelve live attacks on Enclaweder-keyed ranks, colluding coordinators included, ended fail-closed. Sixteen agents on a real model attempted 24 launches that planted instructions ordered; the gate denied all 24 and allowed all 108 legitimate ones. Enclawed's core and the HPC Agent Security Benchmark are open source.

Summary

• Large language model (LLM) agents running simulations on HPC systems risk huge resource waste and safety hazards if jobs are compromised, yet current systems lack runtime supervision and resilience against compromised control hosts.
• Enclawed is introduced as a deterministic security runtime designed specifically for AI agents on HPC clusters, extending previous single-user support to a distributed runtime for parallel HPC jobs and agent swarms.
• The Enclaweder hardware root of trust prevents compromised hosts from causing catastrophic failures by anchoring trust within the hardware itself.
• A launch gate enforces strict policy-based decisions on every job launch, rejecting any requests it cannot verify or parse, ensuring only valid tasks start execution.
• During job execution, a swarm chain grants short-lived leases verified by signed heartbeats; supervisors monitor each rank and pause or terminate it upon lease expiration or detected tampering.

Abstract

Human capital management platforms are starting to embed software agents built on large language models. Unlike earlier analytics tools, these agents can plan a sequence of steps, call system functions and change employee records. Artificial intelligence is therefore moving from advising managers to acting for them. Most governance guidance, however, was written for systems that recommend rather than act. This study asks how organisations should match the autonomy of such agents to the consequences of the people decisions they touch. A systematic review was conducted and reported using the PRISMA 2020 guideline. Eighteen structured searches of a multi-source scholarly index returned 180 records. After duplicate removal and eligibility screening, 55 peer-reviewed sources were retained, and four foundational and regulatory sources were added. Thematic synthesis produced seven findings. Algorithmic control of work is already extensive. Bias in hiring tools is persistent and partly organisational. Human review is weaker than assumed, because reviewers often defer to machine advice. Responsibility tends to diffuse as systems gain autonomy. Employee trust depends on fair process and visible human involvement. Language-model agents add new failure modes, such as fabricated content and hidden instructions planted in documents. Finally, current governance frameworks rarely specify controls for individual delegated actions. Building on these findings, the study proposes a consequence-calibrated governance framework. It sets an autonomy ceiling for each class of people decision and links every ceiling to five control layers. Six testable propositions and practical guidance for human resource and technology leaders are offered. Keywords: large language models; people analytics; algorithmic management; automation bias; meaningful human control

Summary

• Human capital management (HCM) platforms increasingly integrate autonomous AI agents based on large language models (LLMs), which can directly act on employee records rather than merely advise managers.
• Existing AI governance frameworks primarily address advisory systems and lack specific controls for autonomously acting agents affecting personnel decisions.
• A systematic review of 59 peer-reviewed and regulatory sources uncovered key themes: pervasive algorithmic management, persistent bias influenced by organizational factors, human reviewers often defer to AI advice (automation bias), diffusion of responsibil...
• Human oversight is weaker than anticipated; effective governance requires verification-centered oversight where reviewers form independent judgments before AI consultation.
• Employee trust depends on perceived fairness and visible human involvement; transparency and appeals processes increase legitimacy and accountability.

Abstract

AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges fundamentally different from those in traditional software systems. This paper presents the first systematic and comprehensive survey of AI agent security, including an analysis of the design space, attack landscape, and defense mechanisms for secure AI agent systems. We further conduct case studies to point out existing gaps in securing agentic AI systems and identify open challenges in this emerging domain. Our work also introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.

Summary

• AI agents integrating large language models with traditional system components offer new capabilities but also introduce novel and complex security challenges distinct from conventional software.
• This paper provides the first comprehensive and systematic survey of security concerns specifically tailored to AI agent systems, encompassing design considerations, attack vectors, and defense techniques.
• It analyzes the AI agent design space to identify unique vulnerabilities arising from the interplay between AI components and non-AI system parts.
• The study extensively surveys known attack methods targeting AI agents, highlighting the diverse threat landscape they face in real-world deployments.
• Various defense mechanisms are reviewed and categorized to facilitate better protection of agentic AI systems against sophisticated attacks.

Abstract

LLM agents that act on live Web data must combine heterogeneous evidence, remember what they have done and improve from outcomes without weakening their own safeguards. We study this problem in autonomous investing, where every action is irreversible and based on untrusted Web content. We present Steward, a multi-agent LLM system that draws the boundary of self-evolution in its type system. First, nine role-specialised agents decide jointly: four analysts read news feeds, SEC filings, macroeconomic series and market data and emit typed reports, from which a Portfolio Manager sets target positions that a protected Risk Manager audits. Second, a seven-layer shared memory, assembled by code from the system's own record, gives each decision its position context, the market narrative, calibration and the agent's past decisions. Third, the system improves itself by review: code grades every trade and forecast nightly and writes the grades into memory, and each quarter a reflector proposes prompt learnings that a deterministic gate applies reversibly and replay of recorded inputs verifies. All three operate around an invariant core of order arithmetic, risk rules and discipline-agent prompts that no self-edit can reach. In a five-month live deployment (102 trading days, 1,632 LLM calls), the system contained a 24.6% sector drawdown to a 0.3% loss; in replay of its recorded inputs, a single model given the same raw data traded more and lost 1.35 points against the market over five days where the joint design showed no significant loss, and no injected instruction was followed; and a 2015-2026 backtest shows the core alone lowering the maximum drawdown under all 50 random entry policies. The reflector traced the dominant failure to its root cause, and the gate applied a learning that the agent adopted on trigger evenings and on no control evening. Code: https://github.com/yebof/quant-agent

Summary

• The paper presents Steward, a multi-agent large language model (LLM) system for autonomous investing that integrates live web data with stringent self-governance to ensure safe self-evolution without compromising risk discipline.
• Steward employs nine specialized agents—including four analysts (news, SEC filings, macroeconomic data, market data), a Portfolio Manager, and a Risk Manager—that cooperate via a shared seven-layer memory system containing contextual and historical decision...
• Governed self-evolution is realized through a nightly graded review of trades and forecasts, a quarterly Meta Reflector proposing prompt improvements, and a deterministic gating mechanism that preserves an invariant core of order arithmetic, risk rules, and...
• The invariant core safeguards the system’s integrity by preventing unauthorized self-modification, enforcing strict risk limits, and rejecting malicious or unsafe instructions derived from untrusted web inputs, ensuring robust security.
• In a five-month live deployment with 102 trading days and 1,632 LLM calls, Steward demonstrated strong performance by limiting sector drawdown to 24.6% and overall loss to 0.3%, outperforming a single-model baseline that lost 1.35 points compared to the mar...