Key Takeaway
Two simple listener behaviors — how often an agent accepts a known word and how often it accepts a new word — predict whether a group of chat agents will converge on a single name; changing reply randomness moves those rates and can slow or speed consensus by several times.
ON THIS PAGE
What They Found
Measuring two conditional rates (accept-known-word π and accept-new-word ϕ) compresses the messy behavior of language models into a compact diagnostic that explains macroscopic outcomes. Different models fall into three regimes: permissive (high accidental adoption of new words), near-deterministic (almost always correct), and conservative (hesitant to accept known words). Tuning reply randomness shifts models through these regimes, changing consensus time scaling and sometimes crossing a theoretical threshold where fragmentation becomes possible. semantic capability matching pattern
Explore evaluation patternsSee how to apply these findings
By the Numbers
1For the most conservative model (phi3), the acceptance of known words π drops from ≈0.75 at low randomness (T=0.05) to ≈0.30 at high randomness (T=2.0), crossing the theoretical 1/3 ordering threshold while false-acceptance ϕ stays ≈0.03–0.05.
2Consensus-time scaling exponents (t_conv ∼ N^β) vary by model: llama average β = 1.61 ± 0.28 (rising up to ≈2.0 at high randomness), mistral β = 1.28 ± 0.13, and phi3 β = 1.59 ± 0.47 over the tested sizes.
3Temperature sensitivity (fit t_c(T) ∼ A e^{αT}) is model-dependent: llama α ≈ 0.67 ± 0.14 (consensus time grows ≈4× across the tested T range), mistral α ≈ 0.01 ± 0.02 (temperature-insensitive), and phi3 α ≈ 0.43 ± 0.31.
Why It Matters
Engineers building multi-agent systems Dynamic Task Routing Pattern and technical leaders responsible for agent deployment can use the two-rate diagnostic to predict and steer whether agents will reach shared conventions. Researchers studying emergent coordination can use the (π,ϕ) measures as a compact way to compare models and to design interventions (prompting, memory, or controlled randomness).
Key Figures
Fig 1: Figure 1: Schematic of the Naming Game interaction. In the deterministic NG, the decision step is an inventory check ( w ∈ P j w\in P_{j} ); in the LLM-NG it is replaced by the listener’s LLM call at temperature T T .
Fig 2: Figure 2: The two new interaction channels of the LLM-NG that do not exist in the deterministic NG (Fig. 1 ). Top: a missed collapse occurs when w ∈ P j w\!\in\!P_{j} but the LLM answers NO, at rate 1 − π 1{-}\pi ; the inventories are left unchanged. Bottom: a repaint occurs when w ∉ P j w\!\notin\!P_{j} but the LLM answers YES, at rate ϕ \phi ; the listener discards its inventory and adopts the alien word w w . Together with Fig. 1 , these panels enumerate the four channels (TP, FN, FP, TN) into which the LLM-NG interaction decomposes (Sec. III ).
Fig 3: Figure 3: Microscopic conditional rates for N = 150 N\!=\!150 agents at selected temperatures. Top row: consolidation rate π ( t ) = P ( YES ∣ w ∈ P j ) \pi(t)\!=\!P(\text{YES}\mid w\in P_{j}) for (a) llama3.1:8b , (b) mistral:7b , (c) phi3:14b . Dashed line: deterministic NG ( π = 1 \pi\!=\!1 ). Bottom row: repaint rate ϕ ( t ) = P ( YES ∣ w ∉ P j ) \phi(t)\!=\!P(\text{YES}\mid w\notin P_{j}) for (d) llama3.1:8b , (e) mistral:7b , (f) phi3:14b . Dashed line: deterministic NG ( ϕ = 0 \phi\!=\!0 ). Shaded bands indicate seed-to-seed variance. Note the different y y -axis scales in the bottom row, reflecting the order-of-magnitude difference in repaint noise across models.
Fig 4: Figure 4: Drift proxy Δ ( t ) = m π − λ ( 1 − m ) ϕ \Delta(t)=m\pi-\lambda(1{-}m)\phi ( λ = 1 \lambda\!=\!1 ) for (a) llama3.1:8b , (b) mistral:7b , (c) phi3:14b . Positive drift implies net ordering.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreLimitations
Results are based on simulations with three open-weight models and system sizes N between 50 and 150, so asymptotic scaling and sharp phase boundaries may shift at much larger populations. The measured rates depend on prompt wording and single-token reply format; different prompts or multi-turn interactions will change numerical values. A finite simulation horizon produced some runs that did not reach strict consensus, so reported times and exponents are effective estimates rather than definitive asymptotic results. Hallucination Propagation
Deep Dive
A simple microscopic decomposition explains how groups of chat-style language models form shared names. Each listener response is classified by whether the proposed word was already in its inventory and whether the model answered YES or NO, yielding two conditional rates: π (probability of accepting a known word) and ϕ (probability of accepting a new word). Simulating a minimal Naming Game across three models and a range of reply-randomness settings shows that these two numbers capture the dominant mechanisms driving consensus: permissive models get slowed by frequent accidental adoptions (high ϕ), conservative models stall because they refuse correct collapses (low π), and near-deterministic models behave robustly across randomness settings. planning pattern A mean-field reduction to two competing words gives an analytic ordering condition 3π − 2ϕ − 1 > 0 that matches the simulations and explains the three regimes. Empirically, tuning reply randomness moves models along architecture-specific trajectories in the (π,ϕ) plane, changing consensus times and finite-size scaling exponents; for one model, increasing randomness increased consensus time roughly fourfold across the tested range. Practical takeaway: treat reply randomness as a controllable design knob, but measure π and ϕ for your model and prompt to predict whether more randomness will help (break deadlocks) or hurt (introduce repaint noise). Race Condition Failures
Need expert guidance?We can help implement this
Credibility Assessment:
No clear high-profile affiliations and low author h-indexes; arXiv preprint with no citations — emerging/limited information.