Token-Oriented Object Notation (TOON) – Compact, human-readable, schema-aware encoding of JSON for LLM prompts

Tokenize CSS

A promise based streaming tokenizer

Tokenized zip support

TypeScript definition for strtok3 token

Algorithms to help you parse CSS from an array of tokens.

Solve CSS math expressions

Parses and stringifies CSS selectors

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

A tokenzier for Sass' SCSS syntax

Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.

Common token types for decoding and encoding numeric and string values

tokenizer of source code for jscpd

Avatar style for DiceBear

ProseMirror Markdown integration

Token counting and estimation utilities for multiple LLM models

A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.

Claude tokenizer

Encode JSON → TOON (40-60% token savings for LLMs!)

Tokenizes a string that represents a regular expression.