# Can LLMs Reliably Grade Olympiad Proofs? A Study of Mathematical Verification with LLMs

arXiv:2610.36767v1 Announce Type: new Abstract: Large Language Models (LLMs) have made remarkable progress on solving competition-level mathematics, yet their ability to verify natural-language mathematical proofs remains relatively underexplored. This gap is especially pronounced for Olympiad-level proofs, which require meticulous assessment of each claim in a long reasoning chain. Current proof verification thus primarily relies on expert inspection, which is costly to scale. In this paper, we present a systematic study of LLM-based verification in this setting. Specifically, we introduce a human-curated dataset of USAMO 2025 candidate solutions with expert grading, and conduct a broad evaluation of existing LLM verifiers across frontier models. We also propose TROJAN, an iterative self-critique pipeline for generating high-fidelity adversarial proofs at scale. Our comprehensive evaluation covers 20 models spanning seven metrics, two prompt styles, three inference methods, and three prompt templates, showing that state-of-the-art models still leave considerable room for improvement. To narrow this gap, we propose MABGrader, a Multi-Armed Bandit framework that reframes grading as arm selection over the discrete score space. MABGrader outperforms the SOTA ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%. Code and dataset are available at https://github.com/aziksh-ospanov/TROJAN.

## Why It Matters

LLM-based proof verification is still underexplored at Olympiad level, where every claim in a long reasoning chain must be assessed. This systematic study introduces a human-curated USAMO 2025 dataset with expert grading, evaluates twenty models across seven metrics, and proposes TROJAN for generating high-fidelity adversarial proofs at scale. It is relevant to [Scientific Frontiers](../scientific-frontiers.md), [Large Language Models](../large-language-models.md), and [Knowledge and Reasoning](../knowledge-reasoning.md).

## Sources

- rss: https://arxiv.org/abs/2610.36767

*Back to [Recent News](../../news.md) · [Sub-article index](../index.md).*