At a Glance
When multiple code-writing agents work on the same class, vague specs cause a large drop in integration success; writing richer specifications largely fixes the problem.
ON THIS PAGE
What They Found
Coordination between two code agents collapses as specification detail is removed: independent agents often make incompatible internal choices that break integration. A Mutual Verification Pattern handling the coordination gap helps clarify decisions. A single agent handling the whole class degrades more slowly when specs are stripped, creating a clear gap that comes from coordination failures, not just missing information. A fast, syntax-based conflict detector reliably flags mismatches but Reflection Pattern reporting conflicts didn’t improve integration; simply restoring full specifications did. Breaking the gap apart shows about half the loss comes from agents failing to coordinate decisions and half from not having enough information in the spec. A Human-In-The-Loop Pattern approach could help catch edge cases before deployment.
Not sure where to start?Get personalized recommendations
By the Numbers
1Two-agent integration accuracy falls from 58% (full docstrings) to 25% (bare signatures), while single-agent accuracy drops more gently from 89% to 56%, leaving a 25–39 percentage-point coordination gap.
2An abstract-syntax-tree based conflict detector finds structural mismatches with 97% precision at the weakest specification level, without any extra model calls.
3The performance gap decomposes into roughly +16 percentage points from coordination cost and +11 points from missing information; restoring the full specification alone returns two-agent performance to the single-agent ceiling of 89%.
What This Means
Engineers building systems where multiple code-writing agents split work should prioritize richer specifications to avoid integration failures. Platform owners, QA teams, and reliability engineers evaluating multi-agent workflows should treat shared specifications as the main coordination mechanism rather than relying on post-hoc conflict alerts. A Capability Discovery Pattern mindset helps teams identify and align agent capabilities early in development.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
Results come from 51 class-generation tasks with structural stress tests (like lists vs. dictionaries), so effects may vary across other code styles, languages, or larger system architectures. Experiments used two different large language models and repeated runs, but model family and prompt engineering could change absolute numbers. The conflict detector is high precision for syntax mismatches but doesn’t resolve deeper semantic mismatches or design disagreements between agents Hallucination.
Methodology & More
Researchers evaluated how well multiple language-model-based code agents coordinate when independently implementing parts of the same class. They created 51 tasks and dialed specification detail down across four levels: full docstrings to bare function signatures, and introduced opposing structural biases (for example, preferring lists versus dictionaries) to force integration problems. Two setups were compared: two agents each writing parts of the class, and a single agent writing the whole class. Integration accuracy was measured by whether the independently produced pieces fit together correctly.
Findings show a persistent “specification gap”: two-agent accuracy dropped sharply as specs were stripped (58% to 25%), while single-agent performance dropped more slowly (89% to 56%), leaving a consistent 25–39 point gap attributable to coordination failures. A syntax-based (abstract syntax tree) conflict detector achieved 97% precision at the weakest spec level and runs without extra model calls, but simply reporting conflicts did not improve integration. Restoring the full specification alone recovered two-agent performance to the single-agent ceiling (89%). Decomposing the gap attributes roughly +16 points to coordination cost (agents making incompatible choices) and +11 points to information asymmetry (missing spec detail), suggesting these effects are independent and roughly additive. Practical implication: invest in richer, shared specifications as the primary way to make multiple code agents reliable in production.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Single author with low h-index (5) and institutional affiliations that are not top-tier AI labs; arXiv preprint and zero citations — limited established signals.