The Big Picture
An attacker that uses an agent-style query strategy can reconstruct up to 90% of a system’s hidden entity-and-relationship map using surprisingly few queries, showing serious leakage risks for systems that build internal knowledge maps from documents.
ON THIS PAGE
The Evidence
Adversaries that act like exploratory agents — balancing curiosity-driven discovery with focused follow-up queries and using an external memory of what they've already learned — can efficiently rebuild the hidden entity-relationship graph behind retrieval-based question-answer systems. A two-step approach (fast discovery followed by careful filtering using a large language model) recovers most entities and relations while keeping false positives low. The method works across different domains (medical, agriculture, literature) and across multiple graph-backed retrieval systems, and outperforms prior attack strategies under the same query limits. The approach can be framed in terms of Retrieval-Augmented Generation.
Data Highlights
1Recovered up to 90% of entities and relationships from the hidden knowledge map.
2Evaluated across 3 domains: medical, agriculture, and literature.
3Tested on 2 different graph-backed retrieval systems and consistently outperformed baseline attacks under identical query budgets.
What This Means
Engineers building systems that assemble internal knowledge maps from documents should care because these maps can be reconstructed from outside queries, exposing sensitive structure and links. Security teams and product leaders should use these findings to design monitoring, rate limits, and reduced-answer strategies to prevent structured extraction. Researchers working on agent safety and evaluation can use the attack as a stress test for multi-agent trust and agent reliability studies. For researchers, consider how Tree of Thoughts Pattern could inform evaluation design. Also note the role of Agent in conceptualizing these systems.
Not sure where to start?Get personalized recommendations
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreLimitations
Results are from a black-box, budget-limited attack setup and may vary with different system designs, access controls, or stricter query throttling. The defense landscape was not fully explored; simple mitigations (like noisy answers or limiting subgraph visibility) may raise the attack cost but need careful evaluation. Performance numbers come from specific datasets and systems, so reproduction on other domains or custom pipelines may show different outcomes. To frame defensive reasoning, consider guardrails like Memory Poisoning.
Methodology & More
Many modern systems build an internal map of entities and their relationships from a document collection to support multi-step question answering. When the system returns answers that rely on pieces of that internal map, an outside caller can probe the system adaptively to learn about its hidden structure. An effective attacker strategy treats querying as an interactive exploration problem: explore broadly to find novel nodes, then exploit those discoveries with targeted follow-ups, while storing findings in an external graph memory to avoid redundant queries. The described attack framework uses a two-stage pipeline: a lightweight discovery phase that generates many candidate nodes and links, followed by a filtering phase that uses a large language model to clean up and validate the extracted structure. That agent-like approach, guided by a novelty metric and external memory, recovered up to 90% of entities and relations across medical, agriculture, and literary datasets and beat prior baseline attacks when limited to the same number of queries. The takeaway for practitioners is clear: systems that expose fine-grained retrieval outputs can leak structured knowledge even under strict query budgets, so defenses like answer minimization, query-pattern monitoring, access controls, and deliberate perturbation deserve priority in production deployments. For defensive framing, see Chain of Thought Pattern.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
No affiliations listed and authors have low h-indices; arXiv preprint with no citations.