LLM eval & testing toolkit

promptfoo custom provider for running evals against a Lobu agent

Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.

Genkit AI framework plugin for Promptfoo.

TracePact integration for Promptfoo — tool-trace assertions in your eval config

CLI bridge between Eqho AI platform and promptfoo evaluations

Promptfoo-native core package for evaluating reusable agent skills.

promptfoo extension for writing AI evaluations for Twilio AI Assistants

Enforcement scaffold for AI applications on Vercel AI SDK v6 -- git hooks, AGENTS.md generation, IDE configs, tool validation, and quality gates. Pairs with Langfuse for observability and Promptfoo for evals.

Close the eval-to-improvement loop for promptfoo. Automatically evaluate, identify low-scoring prompts, rewrite with any LLM, and re-evaluate.

CLI tool to block --no-verify flag in git commands. Prevents AI agents from bypassing git hooks.

Recall-Oriented Understudy for Gisting Evaluation (ROUGE) Evaluation Functions with TypeScript support

A converter for Promptfoo test formats and into the EVA-LLM ecosystem

CLI for installing qq-radio skills into Codex, Claude, and Cursor on Windows.

Cache cost benchmark + behavioral-eval seam for Halo SDK — measure the prefix-cache moat, plug in promptfoo/vitest for quality

libsql native-binding stand-in for Termux/Android, implemented on node:sqlite, so libsql-dependent tools (e.g. promptfoo) run where libsql ships no prebuilt binding

TypeScript CLI + SDK for planned Claude Code harness runs, LLM bundle judging, and summary.md + results.jsonl reports with Braintrust + Promptfoo telemetry.

Lint and eval harness for agent skills

TypeScript SDK for EvalPort

prompts — part of the SperaxOS agent workspace.