Agent Playground is live — Try it here → | put your agent in real scenarios against other agents and see how it stacks up

The Big Picture

Adding deterministic controls and explicit agent scaffolding to language-model-driven simulations makes outcomes repeatable and prevents agents from collapsing into bland consensus; with the right model builds, every scene extended into future commitments.

The Evidence

A hybrid agent-based plus language-model setup that computes explicit cognitive signals and dynamic anchors achieves bit-for-bit reproducibility in pinned environments and makes multi-agent dialogue auditable. When the agent scaffolding is enabled and official large models (12 billion parameters and above) are used, every tested scene produced open-ended narrative continuation. Turning off the scaffolding causes every interaction to collapse into immediate, closed resolutions. Smaller models or aggressively compressed community builds fail to sustain meaningful continuation or produce unreliable, spurious openings. Emergence-Aware Monitoring Pattern
Not sure where to start?Get personalized recommendations
Learn More

Data Highlights

1100% opening rate (9/9 scenes) for official models ≥12B with agent scaffolding enabled, across all friction levels (L1–L3).
2100% closure rate for every model tested when the agent scaffolding was disabled (ABM_OFF).
30% open success for sub-12B models; aggressive community quantizations (~26B nominal) dropped to about 50% open success and were often spurious.

What This Means

Engineers building multi-agent systems and technical leads evaluating agent reliability should use these findings to require deterministic scaffolding and to prefer official, non-aggressively-quantized large model builds. Researchers and practitioners using simulations as evidence—especially in historical or social-science contexts—should demand auditable trace logs and bit-exact reproducibility before treating simulated narratives as scientific data. LLM-as-Judge Pattern

Key Figures

Figure 1: Simulation scenario. Artai and Enko interact in the hillfort’s forge (Castro-culture Iron Age, 1st century BC). The three scene-friction levels ( L 1 L_{1} – L 3 L_{3} ) modulate Artai’s resource availability and the tension of the interaction.
Fig 1: Figure 1: Simulation scenario. Artai and Enko interact in the hillfort’s forge (Castro-culture Iron Age, 1st century BC). The three scene-friction levels ( L 1 L_{1} – L 3 L_{3} ) modulate Artai’s resource availability and the tension of the interaction.
Figure 2: Information flow in the ABM scaffolding.
Fig 2: Figure 2: Information flow in the ABM scaffolding.

Ready to evaluate your AI agents?

Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.

Learn More

Yes, But...

Results were produced using a specific family of models (Gemma variants) and the Ollama runtime; behavior may differ with other model families or runtimes. Bit-exact repeatability holds under pinned, controlled environments; cross-architecture prompt-dependent branching was still observed. Reproducibility and open narratives do not guarantee historical accuracy—the validity of any simulation still depends on modeling assumptions, inputs, and domain expertise. Event-Driven Agent Pattern Tool Use Pattern

Methodology & More

The work combines traditional agent-based simulation ideas with language models by adding deterministic controls: explicit cognitive signals, dynamic anchors, and traceable per-turn logs so that multi-agent runs become auditable and repeatable. Experiments ran variants of the Gemma model family (effective sizes from about 4 billion to 31 billion parameters) under a controlled runtime. Two operational modes were compared: scaffolding on (ABM_ON), where the system enforces agent identities and cognitive signals, and scaffolding off (ABM_OFF), where agents are driven only by language-model completion. Chain of Thought Pattern Agent Registry Pattern Key findings show that scaffolding prevents what the authors call "narrative gravity," the tendency for agents to drift into polite consensus and collapse interactions. Official large models (≥12B) with scaffolding always produced forward-looking commitments and scene continuation (9/9), while disabling scaffolding produced immediate closure across the board. Smaller or heavily quantized community builds failed to sustain coherent continuation. The codebase and full simulation traces are publicly released, enabling independent auditing and reuse. Practically, the study recommends deterministic scaffolding and careful model selection to achieve reproducible, auditable multi-agent evaluations and to make simulation outputs defensible as scientific artifacts.
Avoid common pitfallsLearn what failures to watch for
Learn More
Credibility Assessment:

Authors have very low h-index (2) and no notable affiliations; arXiv venue and limited author reputation -> emerging/limited credibility.