In Brief
Turning a company knowledge base into a browsable skill hierarchy lets an AI agent explore and combine topics top-down, producing more accurate, grounded answers than plain search-based retrieval.
ON THIS PAGE
Key Findings
Distilling a document corpus into a navigable skill tree and giving the agent full visibility into that structure leads to higher-quality answers than standard retrieval pipelines. An offline compile step groups documents into topic clusters and writes human-readable summaries; at query time the agent navigates those summaries (like folders and indexes) and pulls up the exact documents it needs. That top-down exploration helps the agent connect information across branches for multi-topic questions and reduces reliance on chance keyword matches from flat search. semantic capability matching.
Data Highlights
1Evaluated on a real-world Wix knowledge base of 6,221 support articles using 200 expert-written questions (WixQA).
2Beat five baseline systems (flat retrieval, hierarchical RAPTOR, agentic multi-turn retrieval, and others) to achieve the top quality across all evaluated metrics on WixQA.
3Hierarchy depth scales logarithmically with corpus size (expressed as O(log_p N)), making the navigable skill approach practical as document counts grow.
Implications
Engineers building AI assistants for customer support, compliance, or internal search should care because the approach yields more reliable, evidence-backed answers for complex, multi-topic queries. Knowledge managers and technical leaders benefit when high-value queries demand correctness over raw query throughput, since investing in a compile step improves downstream answer quality. Human-in-the-Loop Pattern
Explore evaluation patternsSee how to apply these findings
Key Figures
Fig 1: Figure 1 : Retrieve vs. Navigate. Traditional RAG passively feeds fixed passages to the LLM. Corpus2Skill distills the corpus into a navigable skill hierarchy that the agent actively explores, backtracks, and drills into to locate evidence.
Fig 2: Figure 2 : Corpus2Skill system architecture. The compile phase (top) embeds documents, builds a multi-level cluster hierarchy with LLM summarization at each level, and materializes the result as a forest of navigable skills. Root clusters map to skill directories ( SKILL.md ), sub-clusters map to group directories ( INDEX.md ), and leaf documents are stored in a separate document store. The serve phase (bottom) uses an LLM agent with pre-compiled skills and two tools: file navigation ( code_execution ) and document lookup ( get_document ).
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Per-query token and compute cost is higher because the agent reads navigation files (skill and index summaries) during interaction, so the approach is best for high-value or high-risk queries. The compilation process is batch-oriented, so frequent corpus updates require incremental re-compilation or risk stale summaries. Hard clustering can route multi-topic documents into a single branch, creating blind spots unless the compilation or cluster assignment is refined. Evaluation-Driven Development (EDDOps)
Methodology & More
Create a browsable map of the corpus up front rather than relying on on-the-fly similarity search. An offline compile phase clusters documents into a multi-level hierarchy, uses a language model to write short summaries for each cluster level, and materializes each cluster as a standardized skill directory (for example, SKILL.md and INDEX.md files). At serve time, an AI agent receives the skill forest and two simple tools: file navigation (to read summaries and index files) and document lookup (to fetch full articles). The agent navigates down promising branches, inspects summaries to decide where to drill deeper, and retrieves only the leaf documents it needs to answer the query. Tree of Thoughts Pattern Agentic RAG Pattern
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Authors affiliated with National University of Singapore (recognized institution) though h-indices are low and it’s an arXiv preprint, matching a solid/recognized-institution rating.