The Big Picture
A human-in-the-loop system that serves expert-approved past cases and audits new AI drafts reached 94% policy compliance while cutting slow responses by about half, making crop protection advice both safer and faster for Ghanaian smallholders.
ON THIS PAGE
The Evidence
A two-part design served recurring farmer questions instantly by reusing expert-approved recommendation templates and routed novel or mismatched cases through a structured retrieval and automated policy check before an expert reviewed them. The cached, expert-verified memory provided sub-second answers for many queries, while the policy audit layer prevented unsafe suggestions (for example, banned pesticides or wrong dosages). Experts validated drafts and their edits were fed back to expand and refine the cache, reducing repeated work. Surveys and an expert calibration step showed high agreement between automated checks and human decisions, and strong user trust in the workflow during pilots in Ghana. policy check.
Not sure where to start?Get personalized recommendations
Data Highlights
1Policy compliance (safety) reached 94% under the architecture's evaluation protocol.
259.6% of queries were served from the expert-verified cache, enabling a 55% reduction in slow (tail) responses.
385.9% of 135 expert-reviewed reports required no substantive edits, showing low human correction burden.
What This Means
Extension service managers, agritech product teams, and engineers building decision-support tools should care because the design shows how to scale expert knowledge safely: reuse expert-approved guidance where possible and gate novel AI outputs with policy checks and human review. Donors and NGOs planning digital extension pilots can use the pattern to increase trust while limiting risky automated advice. Decision-support tools.
Key Figures
Fig 1: Figure 1: Decoupled policy-constrained decision-support architecture (Pezego-HITL). The proposed framework separates generative capabilities from regional policy databases and expert validator workflows. Input queries from mobile clients are processed through a two-stage routing mechanism: (1) queries matching cached, expert-verified advisory templates are immediately served via the Verified-Case Memory (VCM) caching layer, bypassing generation entirely to achieve sub-second execution; (2) cache misses are routed to a schema-aware SQL-grounded Structured Retrieval pipeline, audited against regional safety rules in a Constraint Auditing Layer, and aligned using Critique Multi-Agent loops before human review. Extension Services Officers (ESOs) validate all drafts, and expert corrections are compiled to dynamically update the VCM caching database and prompt structures.
Fig 2: Figure 2: Integrated evaluation protocol workflow. The protocol comprises two primary streams: (A) Online Telemetry (green box) which logs simulated query telemetry to measure execution speed ( P 95 P_{95} latency), database memory reuse, and policy compliance; and (B) Human Feedback (orange box) which gathers survey data from 30 Extension Services Officers (ESOs) and 36 farmers during hands-on training workshops. These input streams feed into parallel evaluation levels: architecture-level validation uses an automated LLM-as-a-judge (calibrated offline against human expert verification decisions) to assess high-throughput compliance and efficiency, while user-level validation evaluates usability, clarity, and trust. The combined outcomes provide a comprehensive assessment of architectural performance (automated Quality and Efficiency) and empirical utility (usability, trustworthiness, and willingness to reuse).
Fig 3: Figure 3: End-to-end latency distribution comparing B4 and Pezego-HITL. The left panel (CDF) shows a consistent left shift for Pezego-HITL, and the right panel (violin/box representation) highlights lower median and tighter upper-tail behaviour.
![Figure 4: Ablation of memory-routing threshold τ \tau in the VCM mechanism. Lower thresholds increase reuse ratio but may risk quality drift (reduced compliance), while higher thresholds preserve strict matching at the cost of reduced reuse and higher latency. The shaded interval ( τ ∈ [ 0.80 , 0.85 ] \tau\in[0.80,0.85] ) marks a practical operating region where policy compliance remains stable, VCM reuse ratio is high, and latency is minimised.](https://arxiv.org/html/2607.13934v1/2607.13934v1/figures/fig06_tau_ablation.png)
Fig 4: Figure 4: Ablation of memory-routing threshold τ \tau in the VCM mechanism. Lower thresholds increase reuse ratio but may risk quality drift (reduced compliance), while higher thresholds preserve strict matching at the cost of reduced reuse and higher latency. The shaded interval ( τ ∈ [ 0.80 , 0.85 ] \tau\in[0.80,0.85] ) marks a practical operating region where policy compliance remains stable, VCM reuse ratio is high, and latency is minimised.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreConsiderations
Evaluation used simulated query telemetry and a calibrated automated judge that matched human experts well but is not a full replacement for long-term field trials. The pilot data come from maize-focused work in one region of Ghana, so results may differ across crops or ecological zones. Connectivity and offline use were not fully solved—client-side caching and offline sync remain necessary for many rural settings. offline sync.
Methodology & More
The system separates two jobs: (1) a verified-case memory that stores past, expert-approved recommendations and serves matching queries almost instantly, and (2) a schema-aware pathway for novel queries where the system translates farmer text into structured checks against regional pesticide and crop-stage rules. If the cache finds a close match (high semantic and structural similarity), the stored template is returned immediately. If not, the system generates a draft recommendation, runs an automated constraint audit against registered pesticides and safety rules, and then routes the draft to a location-matched expert for final validation. Expert edits are logged and used to update templates and search indices so the system improves over time. verified-case memory and schema-aware pathway.
Evaluation combined a simulated workload of 1,240 field queries with a protocol that measures safety compliance, usefulness, latency, and human editing workload. An automated judge (itself calibrated against 135 human-reviewed reports) showed high agreement with experts and enabled high-throughput auditing. Results: 94% policy alignment, nearly 60% of queries served from the expert-verified cache, a 55% drop in slow responses, and most expert reviews needing no major edits. The architecture thus shows a practical way to keep AI assistance both useful and safe by making human expertise central and reusable, while lowering the time experts spend repeating the same guidance. Future work should test more crops, ecological zones, and offline operation in low-connectivity settings. Schema-aware pathway
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
ArXiv preprint with no affiliations and generally low author h-indices (highest ≈7). No strong venue or top-institution signals.