# Suan: Rectifying Direct Preference Safety Alignment in Large Language Models

arXiv:2609.08634v3 Announce Type: replace-cross Abstract: Integrating robust safety guardrails into Large Language Models (LLMs) is essential for delivering helpful yet harmless responses. While proprietary systems exhibit reliable safety controls, their underlying methodologies and trade-offs remain largely undisclosed. Achieving comparable security in open-weight models remains a persistent challenge, as post-trained variants frequently suffer from over-refusal and degraded general quality. To overcome these drawbacks, we introduce Suan, a novel preference optimization algorithm. Unlike existing methods, we formulate the optimization objective directly at the gradient level, bypassing the standard variational derivation. As a result, we obtain more interpretable and robust training dynamics. Extensive evaluations across a diverse suite of competitive baselines and benchmarks demonstrate that Suan achieves superior safety alignment while fully preserving response utility.

## Why It Matters

This matters because it addresses a practical frontier problem: Integrating robust safety guardrails into Large Language Models (LLMs) is essential for delivering helpful yet harmless responses. While proprietary systems exhibit reliable safety controls, their underlying methodologies and trade-offs remain largely undisclosed. The result is significant for the broader AI field, connecting to the encyclopedia's coverage of [large language models](../large-language-models.md) and [training pipelines](../training-pipelines.md). It underscores how ongoing research is reshaping [ai safety](../ai-safety.md) and the questions engineers, policymakers, and researchers will confront next.

## Sources

- rss: https://arxiv.org/abs/2609.08634

*Back to [Recent News](../../news.md) · [Sub-article index](../index.md).*