When to Intervene? An Empirical Study of Errors and Self-Recovery in LLM Agents
On this page
arXiv:2610.36471v1 Announce Type: new Abstract: LLM agents interact with Web services to perform tasks on users’ behalf. Errors during these interactions can disrupt execution and prevent task completion, motivating external intervention. However, agents may self-recover through continued execution, raising the question of when to intervene and when to wait for self-recovery. We present an empirical study of errors and self-recovery in LLM agents. We construct ReInBench, a dataset containing 1,259 agent traces and 1,403 error events across computer-use and Web tasks. Our study examines how often and how quickly agents self-recover, how self-recovery relates to subsequent execution and task completion, and whether self-recovery can be predicted when an error occurs. We find that 725 error events self-recover, often within a few steps, with self-recovery performance differing across Execution LLMs. Self-recovery can also help agents recover from subsequent errors of the same type and support overall task completion. Intervention should therefore prioritize errors unlikely to self-recover, while allowing agents to continue when self-recovery is likely. Building on these empirical findings, we propose ReCoF, a method that predicts whether an error will self-recover to inform whether additional intervention is needed. ReCoF achieves an average relative improvement of 9.02% in Macro-F1 over the strongest baseline on ReInBench. Our code and data can be found at https://method-notes.github.io/ReCoF/.
Why It Matters
LLM agents interacting with Web services can self-recover through continued execution, raising the question of when to intervene and when to wait. This empirical study builds ReInBench with 1,259 agent traces and 1,403 error events across computer-use and Web tasks, finding that 725 error events self-recover often within a few steps and that self-recovery performance differs across execution LLMs. See Agents and Autonomy, Human-AI Interaction, Large Language Models, and AI Safety.
Sources
Back to Recent News · Sub-article index.