AI Safety and Governance

Safety research and institutional governance seek to manage AI’s risks. This sub-article expands Section 3.4 and 4.3 of Philosophy and Section 4 of Recent Topics.

Types of Risk

• Misuse — harmful applications of powerful models.
• Accidents — failures from misaligned objectives or unexpected behavior.
• Existential risk — the possibility that a misaligned superintelligent system causes catastrophic harm.

Safety Research

• Alignment (alignment.md) and interpretability (interpretability.md).
• Red-teaming, adversarial robustness, and capability/risk evaluations.
• Preparedness frameworks in frontier labs.

Governance Instruments

• EU AI Act (2024) tiers obligations by risk.
• National strategies, executive orders, and international summits (Bletchley 2023, Seoul 2024).
• Compute governance, licensing, and export controls.

Open Questions

• How to verify safety at scale?
• Who audits frontier models?
• How do we coordinate globally without centralizing power?

Why It Matters

Safety and governance determine whether powerful AI benefits or harms.

Sources

• Bostrom, N. (2014). Superintelligence.
• European Parliament. AI Act (2024).
• Bletchley Declaration (2023).

Back to Philosophy · See Recent Topics.