CLI tool for evaluating artifacts with LLMs — Swiss Elo ranking, pass/gate, and qualitative review

A tiny JSONL LLM judge

skill-harness engine — spec, discover, run, LLM-judge grade, score, results (internal API)

Score LLM output instead of asserting equality. Deterministic metrics, an optional LLM judge, and the bill for both.

Test/optimize loop for agent skills — run spec'd scenarios on pi, LLM-judge, score, review, re-run

OpenCode plugin that detects prompt injection in tool call outputs using an LLM judge

Governed structured-output LLM pipeline: rate limit → spend caps (per-user + global circuit breaker) → cache → provider-chain failover → Zod validation → usage logging → LLM-judge, with a deterministic mock provider for CI.

Commitment write-gate for AI coding agents: two-tier (deterministic + LLM judge) pre-execution policy. Engine-agnostic core with a DeepSeek Harness (dsh) adapter.

AI/LLM judge matchers for Playwright — toPassRubric/toScoreAtLeast/toMatchImage over Ollama, OpenAI-compatible endpoints (OpenAI/OpenRouter/NVIDIA/Groq/…), and native Claude

LLM-judge service for the @unotest ecosystem: judges free-form text (chat replies, generated content) against a natural-language rubric and returns a structured pass/fail verdict with reasoning. Runs as a small HTTP service (`npx @unotest/judge`) or in-pr

Multi-backend caching for LLM Judge Toolkit

Hey LLM, you okay? — a unified, pyramid-ordered LLM testing CLI for CI/CD. YAML-defined layers (static → exec → http → llm → judge), LLM-as-a-judge gates, and an automated failure-triage protocol (A/B probe) that tells you whether a red test is your promp

AdiaUI generative-UI system — compose strategies, retrieval, the harvested training corpus, and catalog-aware + LLM-judge validation. Emits A2UI protocol messages; pairs with @adia-ai/a2ui (the protocol runtime). Folded from @adia-ai/a2ui-{compose,retriev

Prompt templates for LLM Judge Toolkit evaluation criteria

Evidence-driven /until-done goal loops for Pi with TDD planning, mise verification, and mandatory LLM judge gating.

Multi-judge consensus strategies for LLM Judge Toolkit

An LLM judge routes every Pi agent turn to the right model — fast execution for routine work, smart reasoning for hard problems — with multi-model failover and automatic orchestration.

LLM-as-judge completion plugin for tagma-sdk pipelines

Core types, Zod schemas, and errors for LLM Judge Toolkit

Cost tracking, monitoring, and batch processing for LLM Judge Toolkit