The Big Picture
Unsupervised skill discovery (DataCOPE) extracts reusable analysis procedures from unlabeled agent runs and improves held-out performance — about 9.7% on report-style tasks and 32.3% on reasoning tasks.
ON THIS PAGE
The Evidence
Agents can learn useful, reusable analysis skills by comparing their own exploration traces instead of relying on labeled examples or human feedback. Two verifier types turn unlabeled runs into contrast signals: an adaptive checklist for open-ended reports and an answer-agreement check for fixed-answer reasoning. Iteratively grouping trajectories and distilling the differences produces skills that transfer to held-out tasks and improve final outputs. Skill granularity matters: a diverse set of fine-grained skills (e.g., nine) outperforms very coarse or overly small skill sets.
Not sure where to start?Get personalized recommendations
Data Highlights
1DataCOPE raises mean scores on report-style tasks by 9.71% over strong baselines.
2On reasoning-style tasks, DataCOPE achieves a 32.30% mean improvement over baselines.
3Using all 9 discovered skills on the reasoning benchmark reaches 62.82% accuracy, outperforming smaller skill sets.
What This Means
Engineers building AI agents for data exploration and reporting — they can improve agent outputs without costly labeled data or extra model training. Technical leads evaluating analytics automation — this offers a lightweight path to better, more reliable agent behavior that generalizes to new tasks. Researchers interested in agent self-improvement or evaluation will find the verifier-guided contrastive approach a practical alternative to supervised refinement.
Key Figures
Fig 1: Figure 1 : Supervised skill discovery requires costly data annotation. DataCOPE instead performs unsupervised skill discovery by deriving task-adaptive verifier signals from unlabeled exploration trajectories and distilling them into reusable skills.
Fig 2: Figure 2 : Overview of the DataCOPE framework. The data-analytic agent samples trajectories from an unlabeled exploration set under the current skill, while an unsupervised verifier derives signals and groups trajectories without gold answers or task success labels. The Skill Manager contrasts the grouped trajectories to distill reusable procedures and create or update the skill iteratively. For report-style tasks, the Adaptive Checklist Verifier uses task-specific checklists to score reports and refine both reporting and checklist-generation skills; for reasoning-style tasks, the Answer Agreement Verifier clusters final answers and estimates self-consistency to guide skill refinement.
Fig 3: (a) Checklist-Score Dynamics on Refinement.
Fig 4: (a) Skill Granularity Analysis.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
DataCOPE requires a reasonably diverse set of unlabeled exploration runs; poor or narrow exploration will limit what it can discover. The verifier designs in the work target two common styles (open-ended reports and fixed-answer reasoning) but may need adaptation for other task formats. There is extra compute and coordination cost: generating many trajectories and running verifiers/judges adds overhead compared with running a base agent once.
Methodology & More
DataCOPE is a framework for teaching data-analysis agents new procedural skills from their own unlabeled runs. Instead of needing ground-truth answers or human judgments, it has three parts: (1) an agent that samples many exploration trajectories on unlabeled tasks, (2) an unsupervised verifier that extracts task-dependent signals and groups trajectories into higher- and lower-quality clusters, and (3) a skill manager that contrasts those clusters to distill reusable procedures and add them as an injection at inference time. For open-ended report tasks, the verifier generates a task-specific checklist and scores reports by verifiable coverage, refining the checklist over iterations. For fixed-answer reasoning tasks, the verifier groups runs by final answer and uses agreement (self-consistency) as an uncertainty cue. Evaluated on a report-style benchmark and a reasoning-style benchmark, the discovered skills consistently improved held-out performance: about a 9.7% gain on report tasks and 32.3% on reasoning tasks. The work shows that finer-grained skill sets help: nine discovered skills performed best on the reasoning benchmark (62.82% accuracy), while too few skills hurt transfer. Ablations indicate the unsupervised verifier signals are essential for useful skill distillation. Practically, this approach offers a way to continuously improve agent behavior without expensive annotation or retraining, but it depends on good exploration data and verifier design, and it adds runtime cost for generation and verification.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Includes a senior author with high h-index (Shumin Deng, 39) and other established contributors — stronger credibility despite arXiv venue.