The Big Picture
Agents that automatically create, test, and reuse compact "skills" get better and faster over time: generated skills hit 87.94% success on tasks where they were used, and skill creation worked for 68.6% of evaluated tasks.
ON THIS PAGE
Key Findings
A skill-focused agent loop—where the agent decides to retrieve or create a reusable procedure, runs bundled tests, stores observations, and refines the package—raises accuracy and lowers run time. Automatically generated skills are often Pareto-optimal: they increase task reward while reducing execution time and token use compared to no-skill runs. Generated skills transfer to other agents with only minor accuracy loss and the whole lifecycle is already being used in production tools that let teams share and refine capabilities. Tree of Thoughts Pattern
Data Highlights
1Generated skills achieved 87.94% average success on the tasks where MUSE-Autoskill produced a skill.
2Skill generation succeeded on 68.6% of evaluated tasks (35 out of 51).
3Adding human-authored skills cut median latency by 4–10% across agents; for example, Codex p75 latency dropped 18% (1297 → 1066 seconds).
What This Means
Engineers building multi-step or tool-using agents will find a concrete way to turn repeated solutions into reusable, testable building blocks that reduce repeated reasoning and speed execution. Product and platform leads running agent fleets can use a shared skill registry so fixes and improvements propagate automatically across teams. Researchers studying agent self-improvement get a practical lifecycle (create, test, remember, refine) and evidence it helps in real benchmarks and deployments. Agent Service Mesh Pattern
Avoid common pitfallsLearn what failures to watch for
Key Figures
Fig 2: Figure 2 : MUSE-Autoskill Agent architecture. MUSE organizes skills into a unified lifecycle of creation, memory, management, evaluation, and refinement, enabling agents to generate, refine, and reuse skills with accumulated experience over time.
Fig 3: Figure 3 : End-to-end flow of MUSE-Autoskill. The Master Agent runs a ReAct loop; when a skill is needed it either retrieves one from the Skill Bank or dispatches the Skill Creator to synthesize a new package ( SKILL.md plus optional scripts/ and tests/ ). The Evaluator runs the bundled tests; on pass , observations are appended to Memory and surfaced on later steps; on fail , the Refiner patches the package and re-enters the loop.
Fig 4: Figure 4 : Adaptive context compression over a DAG of ReAct turns. Each turn is a ( plan , action , observation ) triple; the first KEEP_FIRST and last KEEP_LAST turns are always pinned and only the middle is eligible for compression. Top → \to Middle: Level-1 rewrites individually oversized turns in place. Middle → \to Bottom: when no single turn is oversized but the chain is still over budget, Level-2 merges the compressible span into one synthetic node. Original turns remain in the full history (linked by immutable history_prev / history_next pointers), so the trajectory is fully replayable.
Fig 5: Figure 5 : Generated skills are Pareto-optimal: higher reward, lower latency, and fewer tokens than human skills (mean over 35 tasks where MUSE-Autoskill generated a skill, 5 runs per task). (A) Mean reward vs. median per-task latency. (B) Mean reward vs. median per-task tokens. Both panels show MUSE-Autoskill (blue) using its own generated skills and Hermes (teal) using those same MUSE-generated skills with no modification. Open circle = without skills; light fill = with human skills (reference); solid fill = with MUSE-generated skills (ours). Arrows show the shift from without skills to with MUSE-generated skill for each agent; the inline label gives ( Δ \Delta reward ⋅ \cdot Δ \Delta cost). Full per-task numbers are in Appendix J .
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
Results come from a 51-task subset of the benchmark; excluded tasks tended to be harder, so overall performance may be optimistic. Each generated skill was distilled from a single successful trajectory, which can overfit skill content to that path and under-represent more general solutions. Cross-agent transfer was validated in one direction and with limited runs (5 per task), so broader generalization and statistical certainty need more testing. Spiraling Hallucination Loops
Deep Dive
MUSE-Autoskill treats every capability as a reusable "skill": a small, documented package that may include procedure text, scripts, and tests. The agent runs an iterative planning-action-observation loop and, when it needs a capability, it first attempts to retrieve a matching skill; if none exists it synthesizes one from a successful trajectory, bundles tests, and either accepts or refines the package based on automatic verification. Accepted skills are appended to per-skill memory so future plans can leverage accumulated observations. The system also compresses long histories intelligently so agents keep useful context without blowing up input size. Blackboard Pattern A2A Protocol Pattern On a 51-task subset of SkillsBench (Docker-isolated tasks graded by an automated verifier), skills improved accuracy and reduced execution time. MUSE-Autoskill generated skills for 35 tasks and those generated skills produced 87.94% success on those tasks; generated skills tended to be longer and more procedure-focused than human-written ones and uniquely included test suites by design. Skills raise prompt size (more tokens) but often cut overall reasoning and runtime, producing Pareto improvements (higher reward with lower or unchanged execution time). The lifecycle is already used in production components that let teams publish, find, and update skills so improvements propagate across agents and products. Planned next steps are scaling to the full benchmark, testing more backbone models, and validating transfer on independent datasets. Agent Service Mesh Pattern
Need expert guidance?We can help implement this
Credibility Assessment:
Authors have low h-indexes and no notable affiliations or top venue — emerging/limited info.