# Detecting and Mitigating Query-Based System Prompt Leakage Attacks

arXiv:2610.36295v1 Announce Type: cross Abstract: Large Language Models (LLMs) rely on proprietary system prompts to enforce task directives, safety policies, and access controls. These prompts are vulnerable to query-based leakage attacks, where black-box adversaries extract confidential instructions through adaptive, multi-turn interactions. Existing defenses often assess queries independently, missing malicious behavior that emerges across interactions. We present AdvGuard, a model-agnostic framework for detecting and mitigating system prompt leakage. AdvGuard models cross-query relationships using a bounded semantic interaction graph, statistically filters candidate components, and converts them into line graphs for graph convolutional network (GCN) classification. This approach captures multi-hop patterns in query similarity that distinguish coordinated extraction from benign workflows. A lightweight prompt-level detector provides protection before sufficient interaction history accumulates, and the framework supports black-box LLM deployments. In mixed-query evaluations, AdvGuard improves recall by up to 36% over the strongest baseline while maintaining a false positive rate of at most 3.4%. In multi-turn workloads, it achieves 97.23%-99.1% F1 with 100% recall. Leave-one-attack-family-out tests yield 99.98% mean recall and 99.99% mean F1, with no observed false positives. Integrated into the adaptive PLeak and LeakAgent attack loops, AdvGuard limits exact-match leakage to 0.00-0.01 and substring-match leakage to 0.01-0.09 across evaluated victim models. These results demonstrate robust detection of unseen attack families and substantially reduced prompt reconstruction.

## Why It Matters

AdvGuard detects and mitigates query-based system prompt leakage attacks, where black-box adversaries extract confidential instructions through adaptive multi-turn interactions. It models cross-query relationships with a bounded semantic interaction graph, converts components into line graphs for GCN classification, and adds a lightweight prompt-level detector for early protection. It improves recall by up to 36% over the strongest baseline at low false-positive rates and limits exact-match leakage to near zero across victim models. This matters because system prompts encode safety and access controls that must not be reconstructed by adversaries. See [AI Safety](../ai-safety.md), [Large Language Models](../large-language-models.md), and [Agents, Autonomy, and Tool Use](../agents-autonomy.md).

## Sources

- rss: https://arxiv.org/abs/2610.36295

*Back to [Recent News](../../news.md) · [Sub-article index](../index.md).*