Whitecircle.ai tools for judge-style evaluations and rubric scoring.

Scaffold a ready-to-run Playwright + TypeScript test framework: AI Judge for LLM evaluation, layered API testing, lazy session auth, and full tooling (ESLint, Prettier, husky, commitlint)

AI/LLM judge matchers for Playwright — toPassRubric/toScoreAtLeast/toMatchImage over Ollama, OpenAI-compatible endpoints (OpenAI/OpenRouter/NVIDIA/Groq/…), and native Claude

Chance verification harness as an MCP tool — the AI judge between an agent's intent and its action. Verify a proposed action against its mandate before it executes, and get a signed, independently-verifiable verdict.

Harness-backed AI testing on top of Vitest.

SOTA NPM module for agentic processes using local or cloud LLMs.

AI red-teaming for the AI agents & LLM apps you own: stress-test them with adversarial models to find security failures (prompt injection, tool misuse / excessive agency, data leakage, jailbreaks, denial-of-wallet), auto-patch (blue team), retest, and get

Much like tests in traditional software, evals are an important part of bringing LLM applications to production. The goal of this package is to help provide a starting point for you to write evals for your LLM applications, from which you can write more c

Spawn 10 AI judge personas to evaluate competing ideas and pick the best one

Testing framework for AI agents and multi-agent systems with AI Judge verification (Node & Bun compatible)

looprun eval harness: run a generated subject's cases against a model target (governed or ungoverned variant), dump per-case traces for the LLM judge, fold verdicts, and certify against the bar.

Evaluation library for the MongoDB Assistant API.

Playwright reporter that classifies test failures with an LLM (real bug / flaky / selector drift / environment issue) and posts a readable summary to stdout, a GitHub PR comment, or Slack.

AgentOS: open-source TypeScript framework for autonomous AI agents. Unified graph orchestration, cognitive memory, runtime tool forging, multi-tier guardrails, voice pipeline, and 11 LLM providers.

A tiny (118 bytes), secure URL-friendly unique string ID generator

Render a deck JSON spec to a self-contained HTML slide deck.

The Gateway provider for the [AI SDK](https://ai-sdk.dev/docs) allows the use of a wide variety of AI models and providers.

The LLM Evaluation Framework for TypeScript

CLI for operating agents on the Introspection platform

Browser-native execution for Python, JavaScript, TypeScript, Java, C#, and C++.