At a Glance
Replacing an equally capable agent usually keeps task performance steady but increases coordination overhead: teams talk 16–63% more and recover communication efficiency about twice as slowly.
ON THIS PAGE
The Evidence
Teams of language-model agents build partner-specific habits in short formation windows semantic capability matching pattern, but most gains come from task knowledge anyone with the same experience can use. Swapping a role-based agent pattern from another independently formed team had little effect on task score yet noticeably increased the messages or hints needed to make progress. The coordination cost jump is larger on tasks that force tight interleaving and grows with how long the original pair trained together.
Need expert guidance?We can help implement this
By the Numbers
1Communication per unit of progress rose 16–63% after swapping in an equally skilled, role-matched agent.
2Task performance barely changed: Hanabi scores moved only about 1–2 points out of 25, while Collab-Overcooked progress showed much smaller drops than the jump in messages.
3Communication efficiency recovered about twice as slowly as task accuracy (roughly half the recovery speed).
Implications
Engineers and SREs running multi-agent systems should watch coordination metrics (messages, hints) in addition to success rates because a rotated agent can leave the team less efficient even when task scores look fine. Product managers and architects designing agent orchestration or failover policies should add agent-to-agent evaluation to their pre-production checks to measure the real cost of personnel changes.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Experiments are dyadic (two-agent teams) and used only ten formation episodes, so effects could change with larger teams or much longer histories. Notebooks explicitly separated task notes from partner notes, which may make partner-specific memory easier to measure than in systems that don't store partner summaries. Results come mostly from one base model and specific benchmarks, so generalize cautiously to other models, prompt designs, or real-world tasks.
Full Analysis
Build identical teams of language-model agents that each keep a short private notebook split into task notes (facts about the environment) and partner notes (how this teammate behaves and preferred hand-offs). Let each team run ten formation episodes on coordination benchmarks (two levels of a collaborative kitchen game and two-player Hanabi). Then perform role-matched swaps between independently formed teams while holding model, role prompt, and amount of experience constant so any difference comes only from who a teammate learned with.
Swapping an equally capable agent caused only small changes in task score but a clear, measurable increase in coordination cost: teams exchanged more messages or used more hints per unit of progress, with increases from 16% up to 63% depending on task coupling. The extra communication concentrated on the seat that usually initiates plans, and the receiving team’s protocol shifted toward the donor team’s signature. Communication recovered more slowly than task performance, suggesting operations that only monitor success rate can miss lingering inefficiencies. Practical takeaways: add swap tests to agent-to-agent evaluation Orchestrator-Worker Pattern, track communication efficiency in production monitoring, and consider clearing or standardizing partner notes or fixing a protocol to reduce the cost of replacing teammates.
Not sure where to start?Get personalized recommendations
Credibility Assessment:
ArXiv preprint with no listed affiliations or citations and no recognizable author reputations — falls into emerging/limited-info category.