The Big Picture
Pre-built, hierarchical skill libraries (planning, functional, atomic) let weaker agents reuse prior experience and raise task success by about 10% while cutting redundant actions and steps.
ON THIS PAGE
What They Found
Organizing agent experience into three compact skill levels produces reusable, composable building blocks that transfer across models. A fully automated pipeline extracts these skills from successful runs, refines them iteratively, and expands coverage by exploring failure cases and underused tools. When the resulting skill library is plugged into base agents, it improves task success (~10%) and execution efficiency, and the gains hold across different base models. The retrieval strategy uses a high-level planning match plus a rewritten pseudo-plan to select relevant skills without polluting final prompts high-level planning match.
Key Data
1≈10% relative improvement in task success when the SkillX skill library is plugged into base agents.
2Benchmark setup used explicit splits such as 50 training / 150 test for BFCL-v3 and 90 training instances for AppWorld; evaluations ran four independent runs (Avg@4 / Pass@4).
3SkillX was built from 140+ training instances across benchmarks and validated across 3 different base models; iterative refinement and experience-guided expansion produced additional measurable gains and efficiency improvements.
Implications
Engineers building agents that call tools or APIs: they can pre-build a skill library once and plug it into weaker models to reduce repeated exploration and runtime errors. Technical leads and RL/agent researchers: SkillX offers a scalable way to capture transferable experience and boost sample efficiency without retraining whole models.
Explore evaluation patternsSee how to apply these findings
Key Figures
Fig 1: Figure 1 : Claude Skills follow a long-context, progressively disclosed format, which requires a complex sandboxing system and multiple interactions, thereby posing challenges to robust reasoning. In contrast, SkillX adopts a hierarchical, itemized representation that can be stored and retrieved via a lightweight retrieval module and injected into the system prompt in one time, making it easier to transfer across base models.
Fig 2: Figure 2 : SkillX provides an automated, iterative pipeline for constructing a skills library, integrating skills extraction. skills expansion and skills refinement. The skills library is organized into three levels: planning skills, functional skills, and atomic skills.
Fig 3: Figure 3 : Comprehensive Analysis of SkillX . (a) Performance of Multi-skills: Models exhibit varying performance under different skill composition. (b) Execution efficiency of Multi-skills: Jointly composing all skills yields the best execution efficiency. (c) Iterative optimization: Iterative skill refinement further improves performance. (d) Skill expansion strategies: Experience-guided expansion achieves the best on scalability and performance gains. (e) Analysis of Input tokens: Properly balancing input tokens is crucial for controlling inference cost. (f) Analysis of Execution steps: Experience-based learning reduces the number of execution steps.
Fig 4: Figure 4 : AppWorld benchmark case study: Updating Spotify playlist based on roommates’ suggestions. SkillX successfully handles API call sequences (pagination pattern for playlist retrieval) and cross-app integration (integrating Spotify and Phone APIs), while the baseline without multi-level skills fails due to incorrect API call sequences and inability to complete cross-app integration tasks.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
Skills are tied to specific tool schemas, so direct reuse across very different toolsets or domains will require adaptation. The study focuses on tool-using, interactive tasks; pure conversational or non-tool dialogue scenarios may not benefit the same way. Pre-built skills can propagate source-agent biases and may encourage over-reliance unless the library is regularly refined and expanded with diverse experience. tool schemas
The Details
SkillX turns successful agent runs into a compact, reusable skill library organized at three levels: atomic skills (tool-specific usage constraints and patterns), functional skills (multi-step subroutines grounded in tool compositions), and planning skills (how to order and combine functional skills). A strong backbone agent performs rollouts on training tasks; SkillX automatically segments trajectories, distills multi-level skills, consolidates duplicates, and validates quality. The pipeline adds an exploratory phase that targets underused tools and failure-prone behaviors to broaden coverage beyond the initial training set. At runtime, SkillX retrieves high-level planning skills that match the new task, asks the model to rewrite a task-specific pseudo-plan to align retrieval with the current context, and then injects selected skills into the agent’s prompt in a lightweight, single-shot manner. Experiments on long-horizon, user-interactive benchmarks show the plug-and-play library gives roughly a 10% lift in task success for weaker base agents and reduces unnecessary execution steps; iterative refinement and targeted expansion yield further improvements. The library also transfers to other, stronger base models, making it a practical way to share distilled experience across agents without retraining them from scratch. Dynamic Task Routing Pattern
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
One author (Shumin Deng) has a high h-index (~38), indicating strong expertise; although venue is arXiv and many coauthors have low h-indexes, the presence of an established researcher raises credibility to 4 stars.