Key Takeaway
Keeping only single-robot and pairwise interaction terms can make your selector choose plans that deliver far less coverage — sometimes losing over 40% of the map. Measure the delivered outcome of the selected plan, not just how well your score fits the data.
ON THIS PAGE
What They Found
Approximating the delivered-coverage objective by keeping only singleton and pairwise interaction terms often selects a lower-performing multi-robot plan from the same candidate pool. Across seven indoor maps and four-robot teams, the pairwise truncation or a fitted two-additive score chose worse plans on most maps and at several communication ranges. Losses arose mainly because some plans explored areas that never reached the base station under limited communication, a higher-order effect that pairwise scores miss. Fitting pairwise weights reduced but did not eliminate these selection errors, and average reconstruction error did not reliably predict which plans would be misselected. Chain of Thought Pattern.
Explore evaluation patternsSee how to apply these findings
Data Highlights
1Maximum regret observed: 0.427 (42.7 percentage points of map coverage) on env1 at 3 m communication range.
2At the 15 m generation range, the order-2 truncation picked a lower-coverage plan on 6 of 7 maps, with regret ranging from 0.0034 to 0.337.
3Example case: a pairwise-selected plan explored 68.3% of a map but delivered only 45.6%, leaving 22.6 percentage points of explored area undelivered to the base.
Why It Matters
Engineers and teams building or evaluating multi-robot systems should care because approximate objective functions can steer selection toward plans that look good on paper but fail in delivery under realistic communication. Evaluation leads, product managers, and researchers responsible for multi-agent trust and agent-to-agent evaluation should test selectors by measuring the actual delivered outcomes of chosen plans, not only how well approximations reconstruct scores. Consensus Evaluation
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
Results are limited to frozen replay of deterministic trajectories for four-robot teams on one benchmark and a fixed start configuration; larger teams or replanning could change outcomes. The studied pairwise terms are set-function terms over robot subsets, so findings do not directly transfer to every coordination-graph or joint-action formulation. Candidate pools were small (eight plans per map) and generated by the same frontier heuristics, so different plan generators or online planning were not evaluated. State Inconsistency
Deep Dive
A delivered-coverage objective measures what fraction of a map reaches a fixed base station after robots explore and relay information under realistic communication constraints. Evaluating the objective exactly requires measuring outcomes for every subset of robots; for four robots that is 16 replays per candidate plan. Using logged trajectories on seven indoor maps, the study replayed all subsets and compared three selectors: the exact full objective, an exact order-2 truncation (keep only singleton and pairwise Möbius terms), and a fitted two-additive least-squares model. Each selector chose a plan from a fixed pool of eight candidate joint trajectories; regret was measured as the exact coverage difference between the exact selector’s winner and the approximate selector’s pick. Keeping only pairwise interactions frequently changed which plan was selected, especially at finite communication ranges where higher-order overlaps and multi-hop relays matter. In many cases the pairwise score picked teams that explored areas that never reached the base, producing large delivered-coverage losses (up to 42.7 percentage points). Refitting pairwise weights helped in some cases but did not reliably prevent poor choices; moreover, lower average fit error did not predict lower selection regret. The practical takeaway: when using simplified objective functions to rank plans, validate them by simulating selection and measuring the actual delivered outcomes, and consider preserving or testing higher-order interaction effects when communication and relays are important. Mutual Verification Pattern Human-in-the-Loop
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Single-author arXiv preprint with no affiliation or citation/h-index signals — effectively unknown.