Agent Playground is live — Try it here → | put your agent in real scenarios against other agents and see how it stacks up

The Big Picture

A small team of specialized AI agents — a planner, retriever, summarizer, and critic — produces more faithful, query-focused meeting summaries than single-pass systems. They do this by planning where to look in long transcripts, selecting evidence with reasoning distilled from a strong model, and iterating feedback (up to three rounds) to refine the answer.

The Evidence

Breaking the task into roles improves coverage and faithfulness on long, messy meeting transcripts. A planner first maps where relevant topics live /patterns/orchestrator-worker-pattern, a retriever picks evidence guided by distilled reasoning traces, a summarizer writes only from that evidence, and a critic checks for missing or inconsistent bits /patterns/guardrails-pattern. This loop gives consistent gains across both meeting transcripts and long narrative tasks, and keeps the work feasible with mid-sized models by capping planning iterations.
Not sure where to start?Get personalized recommendations
Learn More

Data Highlights

1Evaluated on 2 benchmarks (QMSum and SQuALITY), showing consistent improvements across both meeting and long-narrative query-focused summarization tasks.
2Planning iterations are capped at 3 to balance iterative refinement with compute cost.
3Agent model sizes used in experiments include retrievers at 4B and 1.5B parameters and summarizers at 2B and 4B parameters, demonstrating gains with mid-sized models.

What This Means

Engineers building conversational agents or summary features: use the planner+retriever+critic pattern to locate and verify sparse evidence in long transcripts. Product and technical leaders evaluating summarization systems: expect better faithfulness and query alignment from a multi-role pipeline, at the cost of extra compute. Researchers working on model distillation or multi-agent coordination: the chain-of-thought distillation approach generalizes reasoning behavior across agent roles, aligning with multi-agent coordination.

Key Figures

Figure 1: Single Agent that fails to capture query-specific context, our Multi-Agent pipeline divides tasks between specialized agents. By utilizing reasoning-driven knowledge distillation, the planning and retrieval agents ensure precise content selection, while the critique loop provides feedback-driven refinement for a highly accurate, query-focused summary.
Fig 1: Figure 1: Single Agent that fails to capture query-specific context, our Multi-Agent pipeline divides tasks between specialized agents. By utilizing reasoning-driven knowledge distillation, the planning and retrieval agents ensure precise content selection, while the critique loop provides feedback-driven refinement for a highly accurate, query-focused summary.
Figure 2: Architectural Overview of ReSUM , where the upper part illustrates the training stage with teacher–student distillation and reasoning supervision for the Retriever and Summarizer, and the lower part depicts the multi-agent pipeline coordinating Planner, Retriever, Summarizer, and Critique agents for iterative, query-focused meeting summarization.
Fig 2: Figure 2: Architectural Overview of ReSUM , where the upper part illustrates the training stage with teacher–student distillation and reasoning supervision for the Retriever and Summarizer, and the lower part depicts the multi-agent pipeline coordinating Planner, Retriever, Summarizer, and Critique agents for iterative, query-focused meeting summarization.

Ready to evaluate your AI agents?

Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.

Learn More

Considerations

The approach depends on high-quality teacher models to generate reasoning traces, which can be costly or unavailable in some domains. Iterative planning and critique improve results but add compute and latency compared with single-pass summarizers, which may matter for real-time use. The automated critic reduces errors but cannot fully replace human judgment for subtle semantic mismatches or user-specific preferences. /glossary/reflection

Methodology & More

ReSUM splits query-focused meeting summarization into four cooperating agents: a Planner that scopes the transcript into subtopics and traversal steps, a Retriever trained to select supporting utterances, a Summarizer that writes only from retrieved evidence, and a Critic that evaluates coverage, factuality, and adherence to the plan. Training uses a teacher model to produce step-by-step reasoning traces (chain-of-thought) /glossary/reflection, and smaller student models are fine-tuned to internalize those reasoning behaviors so retrieval and generation become more reasoning-aware. The planner and critic use a stronger model during execution to guide and validate the loop, while mid-sized models handle retrieval and summarization to keep costs reasonable. /in-placeholder 19592? If this were real, this placeholder would be removed. Also see Model Context Protocol (MCP) Pattern for related tooling. On two benchmarks—one focused on meetings and one on long narratives—the multi-agent loop delivered consistent improvements across lexical and semantic metrics compared with single-pass and static retrieval methods. The key practical wins are better targeting of scattered evidence in long transcripts and fewer hallucinations because the summarizer is constrained to retrieved, critiqued evidence. Trade-offs include extra compute and dependency on teacher models; practitioners should weigh these against the need for faithfulness and query alignment and can tune the number of planning iterations (the authors cap it at three) to find a cost-quality balance.
Avoid common pitfallsLearn what failures to watch for
Learn More
Credibility Assessment:

Authors affiliated with an IIT (recognized academic institution) but relatively low h‑indices and arXiv venue; solid but not top-tier pedigree.