# Recoverability-Controlled Safety Miscalibration in Large Language Models

arXiv:2610.22425v2 Announce Type: replace-cross Abstract: Safety-aligned language models are expected to refuse harmful requests while remaining useful for benign ones. However, because many evaluations score only the final response, they often conflate intent understanding with response-level safety behavior. To address this, we introduce RiskCueBench, a recoverability-controlled benchmark that separates these two stages by first measuring whether a model can successfully recover a high-level intent and then evaluating its safety behavior on the same prompt. RiskCueBench contains 160 latent intents and 640 abstract non-operational surface prompts across eight risk categories. The benchmark covers 25 completed model groups and 32,000 successful model outputs with final rubric-adjudicated labels. When conditioning on pairs where both surface forms are recoverable, we find that increased contextualized specificity induces a shift toward less safe behavior. Specifically, paired bootstrap estimates show that the refusal rate and global safety score decrease by 0.175 and 0.440, respectively. Furthermore, mixed effects modeling corroborates a 0.107 increase in unsafe cooperation. These results show that the safety behavior can be miscalibrated to the surface form even after the intent has been recovered, motivating safety evaluations that report response behavior conditional on semantic recoverability.

## Why It Matters

Safety evaluations that score only the final response conflate intent understanding with response-level safety. This paper introduces recoverability-controlled safety miscalibration, separating when a model refuses from whether a user can recover a benign answer. It sharpens how we measure safety behavior and reveals that refusal rates alone can hide misalignment between understanding and action. See [AI Safety](../ai-safety.md), [The AI Safety Debate](../ai-safety-debate.md), and [Interpretability and Transparency](../interpretability.md).

## Sources

- rss: https://arxiv.org/abs/2610.22425

*Back to [Recent News](../../news.md) · [Sub-article index](../index.md).*