Key Takeaway
A small, local assistant can automatically settle about 38% of B2B quoting decisions with no worse accuracy than strict pricing rules, by using model-derived confidence to hand risky cases to humans.
ON THIS PAGE
What They Found
A multi-component assistant running on a single machine settled 38% of quote decisions automatically while sending rule-triggered cases to managers and leaving 19.5% for the salesperson. The system reads confidence directly from the model's option probabilities and uses fixed thresholds to decide when to act versus defer, avoiding the model verbalizing its own confidence. All numeric outputs come from code (not the model text), so outgoing replies never contain unchecked numbers. The tradeoff is more escalations to managers, but no increase in risky or incorrect commitments compared with following pricing rules. This pattern has been explored in the tool-use pattern context.
Explore evaluation patternsSee how to apply these findings
By the Numbers
138.0% of decisions were settled automatically with the fast decision agent
242.5% of decisions triggered hard pricing rules and went to the manager
3Median of 2.67 seconds per automatic decision by the fast agent
Why It Matters
Sales engineering teams and product leaders who need to automate routine quoting but cannot send data to cloud services will find this approach useful because it runs locally and enforces hard rules. AI engineers building practical assistants can reuse the confidence-gating idea and code-first rule enforcement to reduce risky model behavior while boosting throughput. This aligns with the LLM-as-Judge Pattern.
Key Figures

Fig 1: Fig. 1: LoMASA overview. Code computes all numbers and enforces hard rules; System-1 decides when confident; System-2 is consulted otherwise; uncertain cases go to the salesperson. The 27B model also drafts outgoing texts, which may contain only code-computed numbers. (Placeholder figure.)
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreKeep in Mind
Episodes were simulated from public datasets rather than real company logs, so real-world rates of automation and escalation may differ. The system reduces risky mistakes but increases escalations, and the study did not model the manager's time cost for handling them. Results come from one hardware setup and a limited run of the largest model, so performance and latency might change on different machines. Note how this resembles challenges in the State Inconsistency.
Deep Dive
LoMASA is a small, locally run assistant made from two open models plus deterministic code. It splits work into roles: a fast decision agent (System-1) that makes routine calls and reports a numeric confidence read from option log-probabilities, a slower reviewer/writer (System-2) for uncertain cases, an intake agent for parsing messages, and code that computes every number and enforces hard pricing rules so the model cannot override them. Confidence thresholds are chosen on validation data to meet target error rates, so the system delegates uncertain or rule-triggered cases to a human manager or salesperson rather than attempting risky commits. On public benchmarks simulating B2B quoting episodes, LoMASA settled 38% of decisions automatically, left 19.5% for the salesperson, and routed 42.5% to managers when hard rules applied. Automatic decisions took a median of 2.67 seconds, and outgoing customer replies contained only code-checked numbers. The key strengths are (1) training-free confidence gating using model log-probabilities, (2) code ownership of all arithmetic to prevent over-commitment, (3) retrieving similar past cases using numeric features rather than text similarity to improve escalation choices. The main tradeoff is extra escalations; the approach is especially attractive for teams that must keep data local and want a predictable, auditable assistant rather than an end-to-end learned system. The system workflow reflects a Role-Based Agent Pattern and an Event-Driven Agent Pattern.
Test your agentsValidate against real scenarios
Credibility Assessment:
No affiliations or h-index info for authors provided; arXiv preprint and lack of recognized signals -> emerging/limited credibility.