EvaluationExperimental

DataSpace

Name: DataSpace
Rating: 3.0 (35 reviews)
Author: HKUSTDial

by HKUSTDial

Benchmark suite for verifiable analytics by data-focused agents

Python

Updated Aug 5, 2026

Stars

Forks

View on GitHub

What It Does

Benchmarks data-focused agents by measuring verifiable analytics over heterogeneous workspaces. Runs standardized tasks across varied datasets and workspace formats to compare agent outputs, traceability, and reproducibility. Includes dataset suites and evaluation metrics tailored to data-agents and multimodal inputs for analytics-focused scenarios. This aligns with Evaluation-Driven Development (EDDOps) and Role-Based Agent Pattern for structuring data-agent workflows.

Key Benefits

As agents take on data analysis tasks, we need objective ways to compare their accuracy, provenance, and failure modes across messy, heterogeneous data. Until now, evaluations focused on text-only tasks; DataSpace targets data-agents and verifiable analytics so teams can build an agent track record tied to real-world workspaces. That visibility is essential for multi-agent trust and for deciding which agents to delegate sensitive analysis to. This aligns with Agent Registry Pattern to manage agent capabilities and provenance.

Best For

Researchers and engineering teams evaluating and comparing data-analysis agents across diverse datasets and workspace formats. This workflow can be enhanced by applying Orchestrator-Worker Pattern to coordinate diverse data tasks and ensure reproducible results.

Applications

Compare data-agents on accuracy and provenance across heterogeneous datasets
Validate reproducible analytics before deploying agents to production workspaces
Surface agent failure modes and traceability gaps in multimodal data tasks

Avoid common pitfalls

Learn what failures to watch for