Key Takeaway
A small multi-agent pipeline that fixes the model evidence and then plans, writes, and reviews explanations produces audience-specific explanations that are clearer, less templated, and remain faithful to the underlying attributions.
ON THIS PAGE
Key Findings
A structured ExplanationCard holds a single, immutable summary of the prediction and feature attributions, while three agent roles — a planner, a [writer], and a reviewer — transform that evidence into narratives for clinicians, patients, or data scientists. The approach yields explanations that differ in style and content by audience (simpler language for patients, technical detail for data scientists) while preserving alignment with the original attribution scores. Generated text shows lower repetition and higher lexical diversity than single-prompt baselines, and narrative priorities can legitimately shift (for example, patients get actionable features up front). Evaluation used automated metrics, multiple LLM judges, Elo-style rankings, and a human study on two clinical datasets.
Data Highlights
1Far less templated output: XstrAI’s repeated trigram ratio for data scientist narratives was 0.042 versus 0.362 for the worst baseline, indicating much lower repetition.
2Readability split by audience: patient narratives had a Flesch–Kincaid Grade Level of 12.09 vs 15.69 for data scientist narratives (lower score = easier to read).
3Experimental setup used two clinical datasets with Random Forest models (accuracy 0.68 on diabetes, 0.77 on stroke) and fixed SHAP attributions as the shared evidence across all audiences.
What This Means
Engineers building explanation layers or agent-based pipelines will find a concrete architecture for turning fixed model evidence into audience-aware prose. Product leads and clinical AI deployers can use the approach to reduce miscommunication risk by producing patient-friendly messages while keeping developer-facing outputs technically precise. Researchers studying explainability or human-AI interaction can reuse the dual evaluation protocol to separate fidelity from audience fit. The design leverages three distinct agent roles to coordinate narrative production and quality checks.
Not sure where to start?Get personalized recommendations
Key Figures
Fig 1: Figure 1. XstrAI architecture. Each case is represented as a structured ExplanationCard and processed through three stakeholder-specific pathways, each comprising three agents: the Framer produces a narrative plan, the Narrator generates the explanation, and the Reviewer evaluates the output across five audience-aligned dimensions. Rejected outputs trigger a bounded revision loop routed back to the Framer or Narrator. Finally, the generated narratives undergo intra- and extra-narrative evaluation.
Fig 3: Figure 3. Competitive ranking analysis: Elo log-strength (zero-centered; 0 = average system) across three audiences, pooled over 3 judges and 2 datasets. Panels are sorted by Elo score, and percentages indicate pairwise win probability against the average system (e.g., Elo + 1 ≈ 73 % +1\approx 73\% ). Error bars represent 95 % 95\% bootstrap confidence intervals.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Results are demonstrated on two tabular clinical tasks using Random Forest models and SHAP attributions; behavior may differ with other model types, explanation techniques, or non-tabular data. The system depends on large language models for generation and review, so grounding, hallucination, and model updates remain practical risks. Larger, targeted human-subject studies are needed to measure real-world usefulness, trust, and cognitive load across diverse populations. For safety and reliability considerations, teams can apply guardrails and evaluation best practices from patterns like Guardrails Pattern and Evaluation-Driven Development (EDDOps).
Full Analysis
A machine-readable ExplanationCard captures the prediction case, feature attributions, feature metadata, and generation constraints and acts as the single source of truth. For each target audience (clinician, patient, data scientist), three distinct agent roles operate in sequence: the Framer plans what to say and in what order, the Narrator turns the plan into text, and the Reviewer checks the draft against the ExplanationCard and audience constraints, triggering bounded revisions when mismatches are found. Intermediate artifacts are stored for inspection so differences across outputs are due to communicative choices, not changing the underlying evidence. The system was tested on diabetes and stroke prediction datasets using Random Forests and SHAP for local attributions. Against 11 baselines (including single-prompt role conditioning and existing narrative systems), the multi-agent pipeline produced audience-differentiated narratives that were less repetitive, more lexically diverse, and maintained fidelity to attribution evidence. Patient outputs prioritized actionable points over strict attribution rank, while clinician/data scientist outputs preserved technical detail. Evaluation combined stylistic/readability metrics, multi-family LLM judging aggregated by Elo-style ranking, and a human study, providing a clear separation between intra-narrative fidelity and extra-narrative audience appropriateness. The design supports traceability and targeted refinement, but broader validation across models, explanation methods, and larger human cohorts is needed before wide deployment. -> Additional reference to evaluation approach can be guided by Evaluation-Driven Development (EDDOps).
Explore evaluation patternsSee how to apply these findings
Credibility Assessment:
Authors are affiliated with recognized universities (Politecnico di Bari, La Sapienza) though h-indexes are modest and it's an arXiv preprint; solid but not top-tier.