At a Glance
Compose a few learned role vectors inside a single frozen model to get the benefits of many cooperating agents in one pass — matching multi-agent quality while cutting token cost ~20× and improving over single-role steering by about 2.2%.
ON THIS PAGE
Core Insights
A small, trainable module learns a codebook of role vectors (each representing a specialized perspective) and a lightweight router that mixes them per query to create a single internal steering signal. Applying that composed vector to a frozen model produces multi-perspective answers in one inference pass, avoiding the back-and-forth of multiple agents. Across reasoning and personality benchmarks, this approach beats single-role steering methods on average and reaches performance similar to multi-agent systems while using far fewer tokens.
Data Highlights
1Average gain of 2.2% over single-role/single-agent specialization baselines.
2Approximately 20× reduction in inference token cost compared to multi-agent systems that use multi-turn exchanges.
3Evaluated across five reasoning datasets (MMLU, TriviaQA, MATH, GSM8K, MedQA) plus a personality benchmark, showing consistent improvement across tasks.
What This Means
Engineers building conversational or decision-making agents who need multi-perspective reasoning without the expensive multi-turn coordination should consider this — it delivers team-like reasoning in one pass. Technical leaders evaluating deployment cost vs. quality can use this to hit multi-agent quality targets with much lower token and latency budgets. Researchers exploring ways to get diverse reasoning from a single model should see this as a practical middle ground between fixed roles and full multi-agent pipelines.
Test your agentsValidate against real scenarios
Key Figures
Fig 1: Figure 1: Comparison between single agent , MAS and MoRe . MoRe achieves dynamic multi-perspective specialization within single-turn inference, outperforming single-agent methods while achieving higher efficiency than MAS.
Fig 2: Figure 2: Comparison between (a) conventional GRPO and (b) GRPO for MoRe . Our policy is defined on the candidate role space, and the rollout only depends on the composed vector generated from the composer, making the credit assignment clear and optimization efficient.
Fig 3: Figure 3: Performance vs tokens. Optimal value lies at the upper left corner. MoRe matches MAS performance while only consuming tokens of single-agent methods.
Fig 4: (a) Last token embedding.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
The method requires access to a model’s internal activations, so it cannot be used with closed black-box API models. The learned role vectors are latent and not guaranteed to remain human-interpretable, which makes auditing behavior harder. Routing is done per query (not per token or per reasoning step), so very fine-grained adaptivity during generation may be limited without extra cost.
Deep Dive
A frozen large language model can be given team-like, multi-perspective reasoning by inserting a small module that learns and composes internal role vectors. The module maintains a codebook of candidate role vectors (e.g., mathematician, skeptic, engineer) and a lightweight router that computes query-specific weights to mix these vectors into one composed steering vector. That composed vector is added to the model’s internal activations at a chosen layer, nudging the model’s behavior toward a hybrid of perspectives in a single inference pass. Training keeps the backbone model frozen and optimizes only the module via a staged process (supervised fine-tuning followed by a policy-style post-training). On standard reasoning and personality benchmarks, the composed steering approach outperforms fixed single-role steering by about 2.2% on average and matches the results of multi-agent setups while using roughly one-twentieth of the tokens those multi-turn systems require. The approach trades some interpretability (role vectors are latent) and requires model access, but it offers a practical, low-cost way to get collaborative reasoning without running many agents and expensive multi-turn exchanges. hybrid of perspectives in a staged process.
Not sure where to start?Get personalized recommendations
Credibility Assessment:
Authors affiliated with University of Illinois Urbana-Champaign (a recognized institution) and some authors have moderate h-indices (e.g., 16). However the venue is an arXiv preprint and no top-conference publication, so rated as solid/recognized (3 stars).