Key Takeaway
Store evaluated results as typed, reusable evidence (not just scores) and future AI-guided experiments become faster, more reliable, and auditable.
ON THIS PAGE
Key Findings
Turning each experiment into a typed, inheritable lesson and keeping a living formation history lets AI campaigns recombine what worked and avoid what didn’t. Running cohorts of different designs against an external evaluator, then promoting validated lessons into a shared frontier, yields stronger outcomes across many tasks. Praxist produced higher benchmark performance and several domain wins while also delivering an inspectable lineage that explains how each gain was achieved explainability.
Data Highlights
1MLE-bench sweep: 60 medals out of 75 tasks (80.0%), including 49 golds; baseline Claude Code earned 55 medals (73.3%) and 34 golds.
2Rocket case study: found a deterministic landing controller that succeeded 12,288/12,288 trajectories (100%) versus 17.12% for an external optimizer given the same start.
3SLAM case study: replacing always-on visual updates with an observability-aware scheduler cut visual-path processing time by ~72% with no loss in trajectory accuracy.
Why It Matters
Engineers building long-running AI experiments, because the approach makes successful design elements reusable and reduces wasted exploration. Technical leaders evaluating AI tooling, because Praxist produces both better results and an inspectable record (a ‘why this worked’ trail) that supports trust and handoff. Researchers running automated discovery benefit from faster compound progress and clearer reproducibility. See also the Planning Pattern for structuring long-running experiments.
Explore evaluation patternsSee how to apply these findings
Key Figures
Fig 1: Figure 1 : One generation of the Praxist loop. Agents inherit the accumulated evidence and run deliberately different experiments, each producing a reproducible artifact scored by an external evaluator (top). Outcomes become typed claims, pooled into updated evidence and the next generation’s plan: continue, stop, validate, or explore (middle). A lineage records every step (bottom).
Fig 2: Figure 2 : Rocket task overview. Initial states are drawn from three fixed banks—a nominal area-uniform disk, a row-paired slower near-OOD bank, and a faster hard-OOD outer annulus—and are flown by a deterministic hybrid controller: rolling ZEM/ZEV guidance in phase P0, fuel-commit and phase guards, and a P1/P2 terminal-descent corridor. Geometric attitude control feeds a closed-form pitch/yaw box QP that splits each torque demand between gimbal and grid fins, with the RCS restricted to roll. The frozen C05 6DoF plant advances under RK4 at 0.1 0.1 -s steps, and every trajectory is scored once, at the interpolated first landing-leg contact, by a single joint success predicate. The evidence strip records results on the shared task and objective—the 4.03 % 4.03\% starting artifact, Weco’s reported 17.12 % 17.12\% , and the selected controller at 12,288 / 12,288 12{,}288/12{,}288 —and, separately, its full-bank audit ( 40,959 40{,}959 , 40,959 40{,}959 , and 40,960 40{,}960 successes per 40,960 40{,}960 -row bank).
Fig 3: Figure 3 : Quant task overview. Daily market, liquidity, and account state are mapped to 100 allocation scores; a deterministic execution adapter converts those scores into constrained long/cash weights, and orders execute at the next open with per-side costs. The evaluator retrains the policy on each trailing 36-month window, leaves a five-trading-day gap, tests the next quarter, and chains all 28 quarterly returns into calendar-time CAGR.
Fig 4: Figure 4 : SLAM task overview. Synchronized IMU, LiDAR, and camera measurements update one shared ESIKF state in sequence, producing an online 6-DoF trajectory and map. CovSched , the blue gate, intervenes only at the visual pathway: it schedules sparse-direct visual updates by marginal value and admits map points only when they add new geometry, leaving the update order and the sensor residuals unchanged. Evaluation aligns the estimated and reference trajectories and reports full-sequence APE RMSE.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Praxist relies on a task-grounded external evaluator; tasks without reliable automated scoring will be harder to run. The system can be compute- and coordination-heavy because it runs cohorts and maintains lineage metadata, so cost matters for large campaigns. Reported comparisons are against a local Claude Code baseline and task-native controllers; results may vary with different baselines, evaluator noise, or shorter run budgets. Be mindful of Resource Exhaustion risks in large campaigns.
Full Analysis
Praxist treats autonomous research as a generational loop: peers execute reproducible artifacts against pre-specified design contracts, an external evaluator scores outcomes, and those outcomes are interpreted into typed findings (for example: validated parent, fragile candidate, or diagnostic failure). Findings populate a lane-structured frontier and a synthesis step (role-labeled as PI/Chair in their workflow) turns the frontier into an agenda for the next generation — continue, stop, validate, or explore. Durable cross-generation lessons are compressed into 'Gems', and the whole campaign accumulates an active lineage graph that both directs future work and documents how final artifacts were built. See also the Hierarchical Multi-Agent Pattern and Emergence-Aware Monitoring Pattern for how frontiers can be organized and monitored across generations.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
ArXiv preprint with no listed institutional affiliations and mostly low h-index authors (<=5). Limited reputation signals — rates as emerging/limited info.