Scalable Oversight for AI Regulation

Explores computer science research on scalable oversight as a tool for AI regulation that is less dependent on industry self- or co-governance. Discusses scalable oversight as a form of satisfying human-in-the-loop requirements.

Abstract

Frontier AI models form associations that are opaque to humans. This complexity underlies impressive capabilities but also creates risk and undermines reliability. Scalable oversight in model design addresses those issues. It starts from the premise that, as models grow more capable, direct human judgment becomes less effective. In short, human supervision will not scale. The strategy is to enlist AI to oversee AI while keeping humans in control. The key is a hierarchy of models: less capable, more interpretable systems that developers can understand and use to evaluate stronger models.

We contribute to this discussion by asking how law can respond to and capitalize on scalable oversight. We outline what scalable regulatory oversight could look like in emerging regimes. We examine whether scalable oversight can complement or supplement human-in-the-loop requirements, which are now framed as direct human review. Finally, we show how scalable oversight can mitigate the information and resource deficits that have driven regulators to adopt corporate risk management approaches.