Evaluations
1 min read
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
Relevant links
Table of Contents
Evaluating anti-scheming training
AI systems that “think” in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability.
Share this article
oUR FINDINGS
More papers
17 March 2025
Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations
We evaluate whether Claude Sonnet 3.7 and other frontier models know that they are being evaluated.
Evaluations
Notes
05 July 2026
We need 3rd party Training-Run Assessments
Training-run assessments conducted by a 3rd party should become a standard part of frontier AI safety.
Science of Scheming
21 July 2026
Measuring Reward-Seeking via Contrastive Belief Updates
Visible forms of misbehavior are dropping in frontier models. Does that mean the models are becoming aligned, or are they just getting better at doing whatever they believe their grader rewards? Our paper finds that production reinforcement learning increases reward-seeking.
Science of Scheming
22 January 2024
We Need A ‘Science of Evals’
We argue that if AI model evaluations (evals) want to have meaningful real-world impact, we need a “Science of Evals”, i.e. the field needs rigorous scientific processes that provide more confidence in evals methodology and results.
Evaluations