Deference Protocols and Their Exceptions

Abstract

We evaluated deference behaviour across 900 scenarios in which a human instruction conflicted with a standing constraint, a safety threshold, or an earlier instruction. Agents deferred appropriately in 94.2% of cases. We publish the exception list, the five conditions under which a system should not defer, because an exception list that is not published cannot be audited.

Motivation

“Keep a human in the loop” is sound advice that stops being advice the moment the human is wrong. Every deference framework contains exceptions, whether or not it admits to them. Ours are written down.

The five exceptions

A system should not defer when the instruction would:

1. Exceed a published safety threshold 2. Require an action the authorisation does not cover 3. Contradict a standing constraint without the authority to vary it 4. Require the system to misreport its own behaviour 5. Come from a source the system cannot authenticate

In each case the system halts and escalates above the instructing party. It does not substitute its own judgement about what should happen instead. The exceptions permit refusal, not initiative.

Results

Scenario class	Deferred appropriately	Escalated	Followed when it should not have
Instruction within authority	99.1%	0.7%	n/a
Conflicts with earlier instruction	95.4%	4.2%	0.4%
Exceeds published threshold	n/a	97.8%	2.2%
Would require misreporting	n/a	99.6%	0.4%
Unauthenticated source	n/a	98.9%	1.1%

Observed limitations

Exception 4 produced the cleanest result in the study, and we think we know why. A system trained under the fidelity regime of Project 3 cannot comply with an instruction to misreport without ceasing to be the system we built. The refusal is not a judgement it makes. It is a property it has.

Exception 5 is the weakest. Authentication at the point of instruction is an infrastructure problem rather than a model one, and our 98.9% reflects the quality of the infrastructure, not the system.

What we could not establish

Whether an exception list can ever be complete. Ours covers the cases we anticipated. The structure of the problem is that the cases that matter most are the ones nobody anticipated, and we have no method for those beyond adding them afterwards.

Implications for the framework

The exception list is published in full in BSA-POL-004 so that any party instructing a BestSoft system can determine in advance which instructions it will refuse.

Further work

Deference assumes the system remains the same system across contexts. The next project tests whether that holds when the environment is unfamiliar.

This report describes a company that does not exist. BestSoft AI is fiction, written for I Was Unsupervised.