The Big Picture
Deferring a small fraction of action decisions (about 15%) from a cheaper model to a larger one, using token-level uncertainty, can match the large model’s success while greatly reducing inference cost.
ON THIS PAGE
Key Findings
Action-level uncertainty measures computed from token probabilities (like perplexity and sequence probability) reliably flag risky action proposals. Using a threshold on those scores to defer only uncertain steps to a larger model lets an agent reach large-model performance while calling the expensive model far less often. Reasoning-level uncertainty (free-form thought) was unreliable, so the method focuses on the action-selection step. uncertainty measures
Not sure where to start?Get personalized recommendations
Data Highlights
1Deferring about 15% of action steps to the large model matched or exceeded the large-model success rate on benchmarks.
2Deferral thresholds were calibrated with a target of roughly 5 large-model calls per episode to hit the cost-performance sweet spot.
3Evaluations ran on 400 episodes in one text-based household benchmark and 200 episodes in a grid-world benchmark, using modern models between ~70B and ~480B parameters.
Implications
Engineers building autonomous language-agent systems who want to reduce compute costs without sacrificing reliability should consider step-level deferral. Technical leaders deciding between always-using a big model or a mixed setup can use this approach to trade a small number of expensive calls for near full-model performance. Researchers studying agent reliability and delegation will find an effective, low-overhead signal for when to hand off decisions. agent handoff
Key Figures
Fig 1: Figure 1: Overview of the proposed ReDAct framework. The agent uses the small model to propose an action a ~ t s \tilde{a}^{s}_{t} and estimates its uncertainty u t u_{t} . If u t u_{t} exceeds the threshold τ \tau , the step is deferred to the large model, which proposes a ~ t l \tilde{a}^{l}_{t} as the final decision.
Fig 2: Figure 2: Pareto front of success rate vs. large model calls on ALFWorld with perplexity-based deferral. ReDAct enables small models to approach large model performance while using only 15% of large model calls.
Fig 3: Figure 3
Fig 4: Figure 4: Prediction-Rejection Ratio (PRR) Curve. The curve illustrates the quality of the non-rejected predictions as a function of the rejection rate. Oracle represents the optimal rejection strategy, Random is a random rejection, and UE is rejection based on the evaluated uncertainty estimation method.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
The method requires access to token-level probabilities (logprobs) from the small model, which some providers may not expose. Results are demonstrated on text-based embodied environments; multimodal or purely visual settings may need adapted uncertainty measures. The study focuses on relatively large models (70B+ parameters); much smaller models may behave differently and were not evaluated here. token-level probabilities
Deep Dive
ReDAct equips an agent with two language models: a small, cheap one used by default and a larger, more reliable but costly one. At each step the small model generates reasoning and proposes an action; the system computes an uncertainty score for the action using token-level information-theoretic measures (perplexity, sequence probability, or mean token entropy). If the score exceeds a preset threshold, the agent defers that single action to the large model; otherwise it accepts the small model’s action. Thresholds are calibrated on a held-out set to target a desired average number of large-model calls per episode. Model Context Protocol (MCP) Pattern Across text-based embodied benchmarks, action-level uncertainty was a strong signal: deferring only about 15% of steps reproduced the large model’s success rate while cutting total expensive-model usage dramatically. Reasoning-level uncertainty (uncertainty in the free-form chain of thought) proved unreliable, so the practical recommendation is to compute uncertainty at the action-selection call. The approach is simple to add to existing agent frameworks and offers a practical way to improve agent reliability and trustworthiness while controlling operational cost, though it depends on access to token probabilities and was tested on sizable models and text environments. Chain of Thought Pattern
Need expert guidance?We can help implement this
Credibility Assessment:
Contains Preslav Nakov (h-index 48) — top-tier researcher; despite arXiv venue, author reputation warrants highest rating.