EvaluationExperimental

iris

Name: iris
Rating: 3.0 (214 reviews)
Author: syrin-labs

by syrin-labs

Run and verify AI-generated code with reproducible test evidence

TypeScript

Updated Jul 20, 2026

214

Stars

Forks

View on GitHub

Overview

Verifies code produced by AI agents by running tests and collecting executable evidence of correctness. Executes generated TypeScript code in isolated sandboxes, captures failures and test traces, and reports concrete evidence rather than screenshots. Highlights failing assertions, runtime errors, and test outputs to show whether a code-producing agent actually delivered working code. This complements reproducible evidence of correctness and code-producing agents.

Explore evaluation patterns

Methods to assess this tool

Why It Matters

As agents increasingly generate production code, surface-level checks and screenshots are insufficient to judge reliability. Iris gives teams concrete, reproducible evidence of whether an agent's output works, turning transient hallucinations into measurable failure modes. That evidence is essential for building agent track records and continuous agent evaluation pipelines focused on real-world correctness. Leveraging the Model Context Protocol (MCP) helps standardize integration and evaluation across tools and agents.

Ideal For

Teams and researchers validating code-producing agents who need executable proof of correctness and reproducible failure traces. This aligns with approaches in the LLM-as-Judge Pattern to provide authoritative evaluation signals.

How It's Used

Validate code generated by AI agents with executable tests and failure traces
Integrate agent code validation into CI to gate deployments from model-generated code
Collect reproducible evidence for agent track records and debugging of agent failure modes