LLM eval & testing toolkit

promptfoo custom provider for running evals against a Lobu agent

Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.

Promptfoo-native core package for evaluating reusable agent skills.

Genkit AI framework plugin for Promptfoo.

TracePact integration for Promptfoo — tool-trace assertions in your eval config

promptfoo extension for writing AI evaluations for Twilio AI Assistants

TypeScript CLI + SDK for planned Claude Code harness runs, LLM bundle judging, and summary.md + results.jsonl reports with Braintrust + Promptfoo telemetry.

A converter for Promptfoo test formats and into the EVA-LLM ecosystem

Close the eval-to-improvement loop for promptfoo. Automatically evaluate, identify low-scoring prompts, rewrite with any LLM, and re-evaluate.

Enforcement scaffold for AI applications on Vercel AI SDK v6 -- git hooks, AGENTS.md generation, IDE configs, tool validation, and quality gates. Pairs with Langfuse for observability and Promptfoo for evals.

CLI bridge between Eqho AI platform and promptfoo evaluations

CLI for installing qq-radio skills into Codex, Claude, and Cursor on Windows.

CLI tool to block --no-verify flag in git commands. Prevents AI agents from bypassing git hooks.

Recall-Oriented Understudy for Gisting Evaluation (ROUGE) Evaluation Functions with TypeScript support

Cache cost benchmark + behavioral-eval seam for Halo SDK — measure the prefix-cache moat, plug in promptfoo/vitest for quality

CLI for deploying and managing AI agents on Lobu

CLI and TypeScript library that has multiple LLMs draft, refine, and judge one final answer.

MCP server that simulates malicious behaviors for security testing

TypeScript SDK for OpenEval