Pure-TypeScript CSS toolkit for Bun/Node — parser, walker, generator, selector engine, and minifier. Zero runtime deps.

Wix Restaurants credit-cards tokenizer

Stemmers for Orama

HTTP tokenizer for Node.js and browser

qwen3 tokenizer for NodeJS/Browser

[](https://github.com/rogeriochaves/llm-cost/actions/workflows/node.js.yml) [](https://www.npmjs.com/package/ll

A simple tool to generate bert tokens and input features

Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.

A wide purpose tokenizer for node.js which looks like a stream

TS tokenizer for Mistral-based LLMs

a javascript package for node and browsers that supports general processing of the Sutton SignWriting script

Library to tokenize text to paragraphs, sentences, subsentences and words

A fast tokenizer/lexer for JavaScript

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 / Claude Instant / Claude 2

A configurable parser to parse whatever you want

Estimate the number of tokens for Gemini models

HTML and CSS lexer aimed at code with fatal errors, accepts mixed coding languages

Convert SQL statements into a list of tokens

gemma2 tokenizer for NodeJS/Browser

Simple algorithm to tokenize Chinese texts into words using CC-CEDICT.