kotlin-swe-bench

Summary

Provides a Kotlin software-engineering benchmark for evaluating coding agents on realistic SWE tasks. Defines task suites, test harnesses, and scoring to measure correctness and code quality across agent submissions. Focuses on reproducible, language- and task-specific evaluation that highlights strengths and failure modes of coding agents. This repo aligns with planning best practices in benchmarking task suites.

Why It Matters

Best For

Researchers and teams benchmarking coding agents on Kotlin software-engineering tasks and comparing agent performance reproducibly. This work supports scalable evaluation workflows and agent management Planning Pattern and ties into registries and lifecycle management using the Agent Registry Pattern.

Use Cases

• Comparing multiple coding agents on the same Kotlin SWE tasks to establish an agent track record
• Identifying common agent failure modes in code generation and repair for Kotlin projects
• Running reproducible benchmarks to validate improvements from model or prompt changes