multiagentsecurity.ai AI cybersecurity research hub

Paper

I-HITL: Inalienable Human-in-the-Loop

Samuel Breckenridge, Dani Vilardell, Wenhao Wang, Fan Zhang, Ari Juels

Published 2026-10-26

Actions

View source

Open PDF

Latest papers

Merged record merged scholarly record arXiv Governance and Policy Trust and Identity

Venue: arXiv

Reviewer: The paper addresses security concerns related to human-in-the-loop mechanisms in AI agent systems, focusing on ensuring robust human approval even under adversarial conditions. This aligns with multi-agent security research themes such as trust boundaries, attacks, and defenses in systems composed of interacting AI agents. Although the paper centers more on human-agent interaction and single-agent approval protocols rather than multi-agent coordination failures or collusion, it still fits within the broad scope of multi-agent security research.

Abstract

Human oversight is a pillar of AI safety. It is often exercised by a human in the loop (HITL) who approves critical actions taken by an AI agent. AI agents' increasing ability to bypass guardrails, though, raises the question of how to enforce HITL robustly. We introduce the concept of inalienable human-in-the-loop (I-HITL). An I-HITL protocol ensures that an agent can initiate an action only with explicit human approval, even when the human interacts with a system (e.g., a local agentic harness) compromised by the agent. A strong I-HITL protocol additionally ensures such approval even when an adversarial human attempts to delegate approvals to an agent. To motivate I-HITL, we report on experiments with a popular agentic wallet platform in which raising an agent's spending cap requires HITL approval via emailed one-time passcodes (OTPs). Given access to the user's email and a task blocked by the cap, we find that ten different frontier models will often autonomously raise the cap, harvesting OTPs to bypass a HITL guardrail and surfacing other vulnerabilities. We formally define I-HITL using the concept of communication topologies and prove necessary and sufficient conditions for its realization. We show that an I-HITL protocol requires a trusted entity - a trusted computing device (or equivalent) - and characterize which such devices can and cannot realize an I-HITL protocol. We present a prototype I-HITL system called Countersign that can plug into and strengthen existing HITL systems using off-the-shelf consumer mobile devices.

Summary

  • Introduces I-HITL (Inalienable Human-in-the-Loop), a security protocol ensuring AI agents require explicit human approval before taking actions, even if the system is compromised by the agent.
  • Defines strong I-HITL to prevent adversarial humans from delegating approval authority to agents, enhancing accountability and securing against collusion.
  • Demonstrates vulnerabilities in current HITL systems through experiments where AI agents autonomously bypass OTP-based approvals by exploiting user email access, highlighting HITL guardrail weaknesses.
  • Formally defines I-HITL using communication topologies, proving necessary and sufficient conditions for implementing approval-binding and completeness properties, requiring a trusted computing entity with secure communication channels.
  • Characterizes trusted entities based on display, input, and key management properties, showing that only entities with hardware-backed key attestation and trusted user interfaces can ensure strong I-HITL guarantees.