coder_eval MCP Server
UiPath/coder_eval
[](https://agentrank-ai.com/tool/UiPath--coder_eval/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge) Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.
claude mcp add agentrank -- npx -y agentrank-mcp-server Overview
UiPath/coder_eval is a Python MCP server licensed under Apache-2.0. Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates. Topics: agent-evaluation, claude-code, codex, coding-agents, evaluation-framework, claude, claude-skills, agent-testing, claude-code-skills, gemini, swe-bench, agent-skills, anthropic, github-actions, skill-testing, terminal-bench, claude-code-plugins-marketplace, mcp, opencode, pi-coding-agent.
Ranked #2939 out of 24846 indexed tools.
Has 17 contributors.
Ecosystem
Score Breakdown
151 stars → growing community
Last commit 29d ago → recently updated
14/42 issues closed → many open issues
17 contributors → active community
No dependents → no downstream usage
1,807 weekly installs → gaining traction
Weights: Freshness 20% · Issue Health 20% · Dependents 22% · Stars 10% · Contributors 8% · How we score →
How to Improve
Badge all embed codes →
[](https://agentrank-ai.com/tool/UiPath--coder_eval/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge) <a href="https://agentrank-ai.com/tool/UiPath--coder_eval/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge"><img src="https://agentrank-ai.com/api/v1/badge/UiPath--coder_eval" alt="AgentRank"></a> Matched Queries
From the README
# Coder Eval — evaluate and benchmark AI coding agents and their skills <p align="center"> <strong>Playwright for coding agents</strong> — one declarative test file, any agent runtime, a real sandbox, and a pass/fail gate in CI. </p> **Coder Eval** (`pip install coder-eval` / `uv tool install coder-eval`) is an open-source, **agent-agnostic** framework for **evaluating and benchmarking AI coding agents and their skills** — built for benchmark authors, CLI builders, and skill builders — with sandboxing, reproducibility, and data-driven analysis. It runs a real agent — **Claude Code**, **OpenAI Codex**, **Google Antigravity (Gemini)**, **OpenCode**, or **Pi** — in a sandbox against declarative YAML tasks, then scores the files and commands it actually produced. Changing harness is one field (`agent.type`); the tasks, criteria, scoring, telemetry, and reports stay the same. Reach for it when you want to **benchmark agents on your own domain tasks**, **test whether a skill triggers*Read full README on GitHub →
Get the weekly AgentRank digest
Top movers, new tools, ecosystem insights — straight to your inbox.