76 packages matching “cl100k_base”
@aiet/compiler
v1.0.2 · 1 month ago
Deterministic cl100k_base token profiler, priority ranker, and context compiler pipeline for PAKB
No known vulnerabilities
@hypertok/vocab-cl100k
v1.0.0 · 1 month ago
OpenAI cl100k_base vocabulary for hypertok format version 1
No known vulnerabilities
tiktoken-data
v0.1.1 · 3 months ago
BPE tokenizer data files for LLMs (o200k_base, cl100k_base, DeepSeek V3/V2)
No known vulnerabilities
tiktoken-bundle
v0.0.1 · 1 year ago
offline-capable ESM module for cl100k_base tokenization in the browser
No known vulnerabilities
gpt-tokenizer
v4.0.0 · 1 month ago
A pure JavaScript BPE tokenizer with model metadata and pricing for GPT-5, GPT-4o, o-series, and other OpenAI models
No known vulnerabilities
pi-supernova
v0.8.1 · 6 hours ago
CodeMode for Pi and OMP: read, edit, write and bash, with transactional files, source views and shared-input program batches.
No known vulnerabilities
tiktoken-rs-node
v1.0.6 · 1 year ago
_(inspired by [tiktoken-node](https://github.com/ceifa/tiktoken-node))_
No known vulnerabilities
tokenometer
v2.2.0 · 2 months ago
Tokenometer CLI — LLM token cost + latency benchmarking across Claude, GPT-4o, Gemini, Mistral, and Cohere. Multi-format, empirical mode, vision tokens, SARIF output.
No known vulnerabilities
tokenshrink
v2.2.0 · 13 days ago
Prompt compression with tokenizer-aware savings checks and conservative literal protection.
No known vulnerabilities
joplin-plugin-note-categorization
v0.1.12 · 27 days ago
A local-first AI plugin for Joplin that clusters notes semantically, suggests tags and notebook structures.
No known vulnerabilities
@shivam.dixit/token-budget-tiktoken
v0.1.4 · 22 days ago
tiktoken Tokenizer adapter for token-budget, backed by pure-JS js-tiktoken by default (with an optional Node-only native/WASM path).
No known vulnerabilities
serena-slim
v0.0.1-slim.1.10 · 8 months ago
serena MCP (38% less tokens). Quick setup: npx serena-slim --setup
No known vulnerabilities
agentpass-demo-mcp
v1.0.0 · 5 months ago
30 tools vs 1: an agent-payment MCP that measures 230 tokens of schema floor instead of 3,705 (16x smaller; ~48x effective across a multi-hop payment flow). Signed, sanctions-screened, trust-gated payments across x402, Stripe, AP2, MPP, Mastercard Agent P
No known vulnerabilities
@haasstefan/skills-lint
v0.1.2 · 7 months ago
Token budget linter for agent skill files
No known vulnerabilities
@meistrari/gpt-tokenizer
v0.3.2 · 3 years ago
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 / Claude Instant / Claude 2
No known vulnerabilities
@tokn-ai/ffbpe-presets
v0.1.0 · 1 month ago
Lazy-loaded common tokenizer presets for FFBPE
No known vulnerabilities
@skill-tools/core
v0.2.2 · 7 months ago
Core parser, types, and utilities for Agent Skills (SKILL.md)
No known vulnerabilities
tokenmaxxingman
v0.0.21 · 2 months ago
A meticulously deterministic, exhaustively composable, intentionally maximalist tooling artifact for the comprehensive amplification of token expenditure across natural-language workloads — accompanied by a rigorously empirical, corpus-grounded, tokenizer
No known vulnerabilities
@robhowley/pi-structured-return
v1.10.1 · 3 months ago
Turn noisy CLI output into compact structured results (fewer tokens), full logs preserved.
No known vulnerabilities
@tokenectomy/sentinel
v1.0.2 · 1 day ago
Tokenectomy Sentinel — Cut 95% of useless framework noise and token bloat from error logs in Claude & Cursor
No known vulnerabilities