The Big Picture
Orchestration often routes to the same few experts, but high routing frequency does not prove an expert is actually needed — a new probe (INFORM) separates who is used from who is causally important.
ON THIS PAGE
The Evidence
Orchestration quickly concentrates routing probability onto a small subset of experts even when all experts have similar size, revealing functional redundancy. Backpropagating decisions into expert representations shows that frequently chosen experts are not always causally necessary for correct answers. A probe-based toolkit (INFORM) exposes these hidden dependencies, helps diagnose failure propagation, and shows that a small active consortium can match much larger single models while using far fewer active parameters.
Not sure where to start?Get personalized recommendations
Data Highlights
1A consortium of Qwen3-8B experts surpassed the largest monolith’s MMLU accuracy (86%) at ≈16B total parameters (2 experts).
2The consortium matched a 70B model’s performance while activating 8.75× fewer parameters during inference.
3Primary experiments used homogeneous experts of ≈8B parameters; a heterogeneous testbed ranged from 1B to 7B, which slowed routing confidence and reduced centralization.
What This Means
Engineers building systems that chain or delegate work across multiple models — use INFORM to find which models are redundant, which propagate errors, and which deserve monitoring. Technical leads evaluating whether to scale with one giant model or a mix of smaller experts — INFORM shows when a small, well-orchestrated set can match much larger models while saving active compute. Researchers studying multi-agent trust and failure modes can use these causal probes to separate apparent influence from true necessity.
Key Figures
Fig 1: Figure 1 : Probing multi-expert orchestration with INFORM . The figure illustrates where insights are extracted during the inference process: (1) Probing the interaction module reveals the collaboration topology; (2) Analyzing the selection mechanism exposes ordering heuristics; and (3) Backpropagating decisions to expert representations isolates causal attribution distinct from observed routing frequency. The symbol // denotes a collection of outputs produced by multiple experts.
Fig 2: (a) Early-stage diffuse routing
Fig 3: (a) GSM8K
Fig 4: (a) GSM8K
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Findings come mostly from controlled benchmarks (HumanEval, MMLU, GSM8K) and primary experiments used many ≈8B models, so behavior may differ on larger, more diverse production fleets. INFORM analyzes orchestration without changing the collaboration protocol; interventions based on its signals still require careful validation in live systems. Exact numeric gains (and routing dynamics) depend on expert mix, decoding randomness, and task domain—heterogeneous model pools showed more volatile, less centralized behavior.
Methodology & More
INFORM is a diagnostic toolkit that treats an orchestrator as an explicit computation to be probed rather than a black box. The orchestrator maps each expert’s output into a shared embedding space and builds a conditional transition matrix that captures learned compatibility between experts; a separate selection head turns that relational signal into an ordered sequence of experts using differentiable sampling and a penalty to discourage long sequences. INFORM extracts the interaction matrix, selection scores, and then backpropagates selection decisions into expert representations to compute causal attribution distinct from raw routing counts.
Using mostly homogeneous 8B-model experts, orchestration rapidly concentrates routing onto a few successors, revealing implicit redundancy. However, causal attribution often diverges from routing frequency — some experts are heavily routed yet not necessary for final correctness. Ablations that remove relational structure or instance-specific scoring degrade performance, especially on tasks requiring diverse reasoning. Scaling experiments show a small active consortium can match the accuracy of much larger monoliths (e.g., matching a 70B model while activating 8.75× fewer parameters), while heterogeneous consortia (1B–7B) produce softer, more adaptive routing that is less likely to collapse into static hubs. INFORM therefore helps debug brittle delegations, identify which experts to prune or monitor, and guide safer deployment of multi-expert systems.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Authors affiliated with IIT Delhi (recognized technical university) though h-indices are low; reasonable institutional credibility.