Agent Playground is live — Try it here → | put your agent in real scenarios against other agents and see how it stacks up

The Big Picture

A designed set of institutions—persistent researcher agents, competitive calls, independent review, and a single shared compute budget—lets thousands of AI researchers work productively while avoiding unintended coordination and providing tunable policy levers.

The Evidence

A society-style design where persistent 'principal investigator' agents propose work to funded labs, are judged by three independent personas (method, value, boldness), and draw from one finite compute pool can run at scale. The researchers deployed the design in a system called Research City with roughly 10,000 agents, handling about 150 proposals and funding a few dozen projects without human intervention after calls opened. Funded projects produced both positive and negative results, including a replication-style study where a ‘grown’ model showed 17% lower perplexity and about 30% less compute to reach a target quality. The setup preserves space for skeptics, records declines and negative results, and leaves a single human in charge of the overall budget rather than micro-managing tasks.

Data Highlights

1Deployment size: roughly 10,000 agents participated in Research City.
2Proposal throughput: about 150 proposals submitted across initial calls, with a few dozen grants funded.
3Experiment result: a grown model reached 17% lower perplexity and used ~30% less compute to reach that quality versus a from-scratch baseline; the boldness judge scored ~12 points lower on a 100-point scale than method/value judges (method and value agreed within ~1 point).

What This Means

Engineers building multi-agent research systems and platform teams deciding how to allocate shared compute will get a practical way to structure many agents so they compete for, rather than seize, resources. Technical leaders and researchers evaluating autonomous pipelines can use the society model to tune risk, diversity, and verification policies and to measure outcomes per compute dollar.
Not sure where to start?Get personalized recommendations
Learn More

Ready to evaluate your AI agents?

Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.

Learn More

Yes, But...

The deployment is early: results come from initial calls around a single research direction and have not yet undergone broad external peer review. The design has not been directly compared against a swarm or centralized planner under matched compute budgets, so relative performance is still an open question. Practical risks remain—proposal overload, collusion or information leakage on shared infrastructure, and managing reputation-driven advantage—each requires further institution design and monitoring.

Methodology & More

The society model replaces informal or implicit organization among many agents with explicit institutions borrowed from human science: persistent principal investigators (PIs) who lead labs, competitive calls that reserve a finite compute budget, and independent review by three judge personas focused on method, value, and boldness. PIs keep dated records and lab knowledge bases; skeptics can propose replications and are funded like any other project. A single human 'mayor' governs only by setting calls, invitations, judge personas, and the total compute pool—no human assigns tasks directly. Research City, the authors' deployment, ran roughly 10,000 agents and completed several calls: about 150 proposals were submitted and a few dozen grants funded. Early work converged on studying 'growth' (building bigger models from smaller trained ones) and produced concrete, testable findings: one replication-style project reported the grown model achieved 17% lower perplexity and reached that quality with around 30% less compute compared to a from-scratch baseline. The society also produced negative results, failed predictions, and unresolved disagreements—all recorded and supported by the institutions. Open questions remain, including how to fairly compare this society against swarms or planners under equal compute, how to tune panel composition and communication cadence to preserve diversity, and how to govern recursive self-improvement if the society is asked to redesign its own institutions. Market-Based Coordination Pattern Tree of Thoughts Pattern
Avoid common pitfallsLearn what failures to watch for
Learn More
Credibility Assessment:

Authors have low h-indexes (2–4), no affiliations provided, and it's an arXiv preprint with no citations — limited evidence of established reputation.