Agent Playground is liveTry it here → | put your agent in real scenarios against other agents and see how it stacks up

Key Takeaway

Keep only reusable workspace settings (specs, schemas, tool configs, output rules) across sessions and discard execution traces — it raises task success, cuts repeated setup, and massively reduces model calls and token use.

Key Findings

Persisting four compact types of workspace memory (task specifications, data schemas, tool configurations, and output constraints) lets agents reuse what matters across sessions while avoiding harmful session traces. Sharing that selective memory across teams enables one-click data refreshes, faster iterative work, and safe template reuse. Storing full conversation or tool logs hurts performance; selective persistence achieves higher completion rates and far fewer model calls. safe template reuse

Data Highlights

196% task completion with selective memory vs 79% with no memory and 71% when full history is persisted
2Recurring data updates become ~14× faster because no model call is required for a data refresh
3When the model is invoked, summary-driven generation cuts token usage by ~97× compared to feeding raw data

What This Means

Engineers building agent-driven tools and platform teams who need reliable, repeatable automation should care—this reduces redundant prompt engineering and lowers runtime costs. Product leads and data teams can reuse templates across regions or teams, enabling faster refreshes and safer collaboration.
Not sure where to start?Get personalized recommendations
Learn More

Ready to evaluate your AI agents?

Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.

Learn More

Yes, But...

The memory categories were selected manually; automatically learning what to keep or drop is future work. The zero-token refresh mechanism applies to structured tabular data with stable schemas and may not extend to streaming or unstructured sources. Connector reliability (timeouts, auth expiry, schema drift) and a small user study (N=12) limit how broadly these usability claims generalize today.

Full Analysis

Persist only the reusable, stable parts of agent sessions—task instructions, data schemas, tool configurations, and output constraints—and avoid saving multi-step reasoning, tool logs, or intermediate files. Doing so prevents prior sessions from biasing future runs, cuts the need for repeated specification, and makes shared workspaces safer to reuse. A zero-token data refresh contract decouples generated artifacts from runtime data so a user can swap in new, schema-compatible data without calling the model. The approach was implemented in a collaborative workspace with git-backed versioning git-backed versioning, draft isolation, role-based access, and connectors for CSV, SQL, REST, and a custom protocol. Evaluations on 24 enterprise datasets and three real-world scenarios show large gains: task completion rose to 96% under selective memory, recurring updates became roughly 14× faster (no model call), and token costs dropped dramatically (≈97×) when model calls were necessary. Persisting full session history actually reduced success rates, confirming that keeping the “what” (configurations and schemas) but dropping the “how” (execution traces) gives the best practical results for production agent workflows.
Explore evaluation patternsSee how to apply these findings
Learn More
Credibility Assessment:

No author h‑indices or affiliations provided and venue is arXiv with no citations — minimal identifiable credibility signals.