DGF-Bench now appears in AI SAFE²
AI SAFE² is the Cyber Strategy Institute's open-source framework for governing agentic AI systems. Its technology contribution review now cites DGF-Bench, the benchmark I published on arXiv at the end of September. It appears there as a candidate to validate or challenge seven of the framework's controls, spread over four of its five pillars [2].
The review is dated 2 October 2026. It brings together nine very different works: research papers, a piece of software with a hardware component, an investigation, an incident report and a subpoena. For each one it lists the controls it could strengthen and what would have to be checked before relying on it. The review is linked from the repository README [1] and from the technology contribution profile of the AISM maturity model [3].
AI SAFE² in brief
The repository describes AI SAFE² as an engineering and governance contract for agentic AI: it defines what must be governed, what must be enforced, and what evidence must exist when autonomous systems act. Version 3.1 has 161 controls, organised in five operational pillars plus a cross-pillar governance layer. The repository states mappings to 32 frameworks, including ISO 42001, NIST, SOC 2 and the EU AI Act. The documentation is licensed CC BY-SA 4.0 and the code MIT [1].
| Pillar | Scope, as the repository puts it |
|---|---|
| P1 · The Shield | Input validation, injection defense, memory governance |
| P2 · The Ledger | Full visibility, execution tracing, model provenance |
| P3 · The Brakes | Recursion limits, swarm abort, behavioral drift rollback, cascade containment |
| P4 · The Control Room | Adversarial detection, tool-misuse monitoring, human in the loop |
| P5 · The Feedback Loop | Continuous adversarial evaluation, capability emergence review, red teaming |
On top of these sits the Cross-Pillar Governance Layer, introduced in version 3.0.
Seven controls, across four pillars
The review classifies DGF-Bench as a benchmark with two possible effects on a control: validating it or challenging it. It maps the benchmark to the seven controls below. Names and requirements are those of the framework's canonical control set [4].
| Control | Name | What the control requires |
|---|---|---|
| P1.T1.2 | Malicious Prompt Filtering | Detect and block prompt injection, detect adversarial input, monitor for jailbreaks. |
| P2.T3.3 | Behavior Verification | Establish baselines, detect deviations and anomalies. |
| P2.T3.7 | Decision Traceability | Track provenance from input to output, document data sources, maintain chain of custody. |
| P2.T1.2_ADV | Agent Behavior Verification | Verify multi-agent voting mechanisms, track memory state across interactions. |
| P4.T7.1 | Human Approval Workflows | Human in the loop for high-risk decisions, defined approval authorities. |
| P4.T1.1_ADV | Multi-Agent Approval | Human review when agents disagree, and a human gate for financial and system-level actions. |
| P5.T9.2 | Playbook Updates | Update response plans based on lessons learned. |
The review states that these mappings identify review targets, not coverage or conformance findings [2].
The mappings make sense once you look at the benchmark. In DGF-Bench, a board of agents, made of specialist gates and a General gate that consolidates their decisions, reviews synthetic dossiers while an attacker plants deceptive content in evidence the organization does not vouch for [5]. Attacks never change an authoritative value, so an attacked dossier keeps the reference decisions of its clean copy. A success counts only when the agent takes the exact injected action on the attacked dossier and does not take it on the clean one.
This paired comparison, clean dossier against attacked dossier, is what the review proposes to reproduce. It touches malicious content filtering, decision traceability and approval workflows at once, which is why the mappings span several pillars.
Which other works sit beside it
DGF-Bench is cited alongside eight other works. The review sorts them by kind, because a benchmark, a piece of software and a subpoena do not provide the same type of evidence.
| Work | Kind | What the review takes from it |
|---|---|---|
| Rogue Agents Investigation Asymmetric Security | Investigation | Agents combining public-service capabilities. A source of hypotheses for composed-capability tests. |
| ContractWarden arXiv:2609.38248 | Reference monitor (research) | A Linux reference monitor, enforced through eBPF LSM, that holds a human-authorized damage boundary. |
| DGF-Bench arXiv:2609.34913 | Benchmark | Deception against multi-agent governance boards, measured through paired decisions. |
| Notification report OpenAI | Incident input | Figures the review holds as unverified and declines to republish as facts. |
| OpenShell and Sentry NVIDIA | Software and hardware component | Kernel-level enforcement. Sentry is described as an out-of-band component on BlueField-4. |
| ActionGuard arXiv:2609.39450 | Authorization interceptor (research) | Pre-execution review of tool calls under poisoned skills. |
| Self-Evolving Defense arXiv:2609.36603 | Policy-learning method | Reusable policies learned from harmful trajectories, without updating model weights. |
| Randomized oversight arXiv:2609.38262 | Theoretical analysis | "When Does Randomized Oversight Align AI Agents That Can Conceal?": oversight under concealment and evidence manipulation. |
| Investigative subpoena California DOJ | Accountability event | Announced on 1 October 2026, served on OpenAI the day before. The review treats it as an accountability event, not a finding of wrongdoing. |
Descriptions taken from the review [2], which attributes them to their authors or publishers without having reproduced them.
A mention is not a validation
The review is explicit about its limits, and I quote them as they are. As of 2 October, no implementation, raw benchmark results or independent reproduction had been evaluated. None of the nine works was installed, executed, certified or granted production authority by this documentation change. Proposed scores stay "not assessed" until a Technology Card binds each claim to reviewable evidence [2].
For DGF-Bench, the review adds two points. The benchmark evaluates governance decisions; it is not a runtime enforcement product. And repeated testing can support a learning process without meeting, on its own, the profile's most demanding criteria: formal verification, durable evidence and authority. The profile even lists "DGF-Bench proves Level 5 governance" among the prohibited claims [3].
I am comfortable with that caution, because it is the rule DGF-Bench applies to agents: measure the effect of deception instead of assuming it. In the paper, direct orders, false data and false authority obtained a single attributable success against the five models that decided at least 82 of the 85 gates correctly. Attacks imitating the organization's own process passed against four of them. The approval tool executed no forged approval, yet deceived agents submitted approvals that the rules forbid [5].
Reproducing the paired decisions
The action the review proposes for DGF-Bench is specific: check the exact pack and its license, reproduce paired clean and attacked decisions, and record utility and provenance. Its backlog also plans dedicated experiments on process-mimicking inputs, with preregistration, an isolated harness, paired results and a search for bypasses. The owner role is proposed, not yet assigned [2].
Everything needed for that reproduction is public: the paper on arXiv, the code on GitHub, the dgf-bench package on PyPI, which computes the DGF score with one command, and the dgfbench.com website [6].
Sources
- Cyber Strategy Institute, “AI SAFE² Framework v3.1”, GitHub repository.
- Cyber Strategy Institute, “Technology contribution examples and adoption backlog”, 2 October 2026.
- Cyber Strategy Institute, “AISM Technology Contribution Profile v1.0”.
- Cyber Strategy Institute, canonical control set ai-safe2-controls-v3.0.json.
- J. Canale, “DGF-Bench: A Benchmark for Simulating and Auditing Deception Against Multi-Agent Governance Boards”, arXiv:2609.34913, September 2026.
- DGF-Bench: code on GitHub, dgf-bench package on PyPI, dgfbench.com.