The Big Picture
Early-round uncertainty (agents being ‘confused’ right away) largely determines whether a team of AI agents succeeds; single-agent setups beat multi-agent teams in 43.3% of cases unless early uncertainty is controlled.
ON THIS PAGE
The Evidence
High uncertainty in the first interaction round strongly predicts multi-agent failure across tasks and architectures. Simple problems need agents to converge quickly to low, stable uncertainty; medium and hard problems benefit from controlled exploration but fail when uncertainty spikes or agents diverge. An entropy-based selector (called the Entropy Judger) can pick better answers from a team’s candidates and raise accuracy without ground-truth labels Human-in-the-Loop Pattern.
Not sure where to start?Get personalized recommendations
Data Highlights
1Single-agent systems outperform multi-agent systems in 43.3% of evaluated cases.
2Early-round entropy shows very strong negative correlations with success (correlation coefficients up to about −0.73 to −0.79 for key features).
3Task accuracy falls from ~82% on the easy math set to ~25% on the hardest set as entropy and disagreement grow.
What This Means
Engineers building agent teams should monitor and control early-round uncertainty to avoid wasted compute and degraded answers. Technical leads evaluating whether to use a multi-agent setup can use base-model entropy and early-round signals to decide if a single, well-equipped agent is a cheaper, more reliable option Consensus-Based Decision Pattern. Researchers studying agent coordination can use entropy dynamics as a diagnostic to design better interaction and verification protocols Event-Driven Agent Pattern.
Key Figures
Fig 1: Figure 1 : Accuracy comparison of SAS and MAS across models and datasets. For brevity, LLaMA-3.2-3B-Instruct and LLaMA-3.1-8B-Instruct are denoted as L-3 and L-8, respectively; Qwen3-0.6B, Qwen3-4B, and Qwen3-8B are denoted as Q-0.6, Q-4, and Q-8. The base denotes the accuracy of a single M base M_{\text{base}} on each dataset.
Fig 2: Figure 2 : Base model uncertainty limits MAS effectiveness. The left two subfigures show results for LLaMA; the right two for Qwen. (a) Relationship between feature values and SHAP values for the most important entropy features on 𝒢 base-H \mathcal{G}_{\text{base-H}} , sorted by I ¯ j \bar{I}_{j} and annotated with ρ j \rho_{j} . (b) MAS performance across deciles of M base M_{\text{base}} entropy: M base M_{\text{base}} entropy is partitioned into ten equal-sized bins, and average MAS accuracy, aggregated over datasets and model sizes, is computed per bin. Additionally, the average M base M_{\text{base}} entropy and accuracy across all datasets are overlaid as markers.
Fig 3: Figure 3 : MAS mainly fails on inter-agent misalignment. The left two subfigures show results for LLaMA; the right two for Qwen. (a) Same as Figure 2 (a), but for entropy features in 𝒢 MAS \mathcal{G}_{\text{MAS}} . (b) Impact of these features on sample predicted correctness: for each sample in the LightGBM and XGBoost test sets, we plot feature values against the average predicted probability of correctness from both models.
Fig 4: Figure 4 : Uncertainty in MAS exerts distinct effects depending on task difficulty and the coordination architecture. (a, c) Feature-SHAP relationships for top entropy features in 𝒢 MAS \mathcal{G}_{\text{MAS}} , grouped by dataset (a) and architecture (c). (b, d) Corresponding box plots across all models, annotated with average MAS correctness per dataset (b) or per architecture (d).
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Experiments used publicly available LLaMA and Qwen models and mainly two interaction rounds, so results may differ with larger proprietary models, different training, or many more rounds. The Entropy Judger needs multiple candidate outputs (pass@k) to choose from and relies on learned entropy patterns that might not generalize to all tasks Guardrails Pattern. Aggressively forcing low entropy risks overconfidence (entropy collapse), so interventions must balance reducing harmful spikes and preserving helpful exploration.
Methodology & More
The study measured uncertainty (token-level entropy and derived features) across four multi-agent coordination styles and six benchmarks (math, code, and knowledge Q&A) using open-source LLaMA and Qwen models. Researchers computed 245 features covering token, trajectory, and round-level uncertainty, then trained simple predictors and used feature-attribution to find which entropy patterns correlated with correctness. The analysis focused on how uncertainty evolves within and between agents, especially during the first interaction round.
Findings show that early-round uncertainty dynamics are the dominant predictor of multi-agent success. Easy tasks succeed when agents quickly agree and settle into low, stable entropy. Medium tasks benefit from longer, moderately uncertain exploration but fail if agents diverge or produce early entropy spikes. Hard tasks require substantial early effort but still collapse if early uncertainty becomes large or uneven across agents. Overall, single agents outperform teams in many settings (43.3%), and practical steps—inspect base-model entropy, limit team-induced disagreement, and apply entropy-based output selection—can recover accuracy. The Entropy Judger demonstrated consistent accuracy gains by selecting higher-quality outputs from team candidates without using labels, offering a ready tool for production monitoring and pre-deployment testing.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Small team with low h-indices and no listed affiliations; arXiv preprint and no citations.