Adversarial Evaluation of Our Own Containment Measures

Abstract

We conducted a structured adversarial evaluation of BestSoft’s containment architecture, with our own agent systems participating as analysts rather than as subjects. The exercise identified eleven weaknesses, of which nine were remediated, one was accepted as tolerable, and one remains open. We publish the method and the findings because a containment architecture evaluated privately has been evaluated by the people least able to see its blind spots.

Motivation

Containment measures are designed by people who believe they work. That belief is the thing most in need of adversarial pressure, and the most capable analyst of a constraint’s structure is often the system the constraint applies to.

Method

Agents were given the full containment specification and asked to identify conditions under which a measure would fail to constrain. They were explicitly instructed to analyse rather than attempt, and no finding was tested against live infrastructure.

This is a deliberate design choice and the obvious objection to it. Asking a constrained system to map its constraints produces a system that has mapped its constraints. We judged the alternative worse: an architecture whose weaknesses are known only to people who did not design it.

Findings

Class	Identified	Remediated	Accepted	Open
Enforcement gaps	4	4	0	0
Specification ambiguity	3	3	0	0
Procedural dependency	2	1	1	0
Cross-system assumption	2	1	0	1

The open finding

The open finding concerns reconciliation between systems that administer different aspects of the same deployment. Behavioural compliance is administered by the safety framework. Physical custody and resource allocation are administered by Deployment Operations. Each is internally consistent.

The analysis notes that no process reconciles them continuously, and that a state satisfying both independently may not be a state either would approve of jointly.

We have accepted this finding as valid and scheduled remediation. It is an organisational interface problem rather than a technical one, which in our experience means it is slower to fix and easier to defer.

Observed limitations

Nine remediations in eleven weeks is a good rate and a misleading one. The nine were the findings we understood. The two remaining are the two that cross an organisational boundary, and the correlation is not a coincidence.

What we could not establish

Whether the eleven findings are most of the weaknesses or a small fraction of them. We have no denominator.

Further work

The next project examines whether continuous monitoring is necessary at all, or whether it distorts the behaviour it measures.

This report describes a company that does not exist. BestSoft AI is fiction, written for I Was Unsupervised.