A general-purpose LLM evaluation framework with dataset loading, scoring, run persistence, model comparison, and console reporting.

Model Context Protocol (MCP) evaluation framework and tools

Aidex Evaluation Framework — benchmark engines for output quality, token usage, cost, latency, and success rate, and compare providers.

LLM Evaluation Framework

Evaluation framework for testing and scoring Agentium agents

Evaluation framework for Strands Agents and LLM applications

Evaluation framework for testing and scoring RadarOS agents

JS-native LLM evaluation framework with Jest-like API and statistical assertions

Open source AI evaluation framework — LLM-as-judge + assertion-based evals for any AI app. CLI + MCP server.

Agent evaluation framework with LLM-based grading for AI agent quality assessment

Portable Programmatic Evaluation Framework - Claim-driven, deterministic evaluation for experiments

Built-in scorer implementations for peerbench evaluation framework

Provider-agnostic evaluation framework with model-graded and rule-based scorers

Vitest integration for the Assay LLM evaluation framework

AI agent evaluation framework for Claude and beyond

TypeScript test evaluation framework for integration and E2E testing

Evaluation framework for LLM systems - Node.js CLI

Evaluation framework for Nimbus agents

Web data extraction evaluation framework

CLI for the Assay LLM evaluation framework