328,660 packages matching “Text tokenizer”

@inferock/measure

v0.2.4 · 16 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

TypeScript primitives for LLM token usage, billing integrity, cost, reliability, and failure receipt measurement.

89Downloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.2.4 (the latest release), from the OSV.dev database.

@orama/tokenizers

v3.1.18 · 8 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Additional tokenizers for Orama

45.4KDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.1.18 (the latest release), from the OSV.dev database.

@yarflam/potion-base-8m

v1.0.4 · 4 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Fast Model2Vec inference for potion-base-8M embeddings without ONNX or heavy ML frameworks

895Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.4 (the latest release), from the OSV.dev database.

@lenml/tokenizer-gemma

v3.7.2 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

gemma tokenizer for NodeJS/Browser

186Downloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.7.2 (the latest release), from the OSV.dev database.

bpe-lite

v0.5.2 · 5 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Offline BPE tokenizer for OpenAI, Anthropic, and Gemini — zero dependencies

8.5KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.5.2 (the latest release), from the OSV.dev database.

happynodetokenizer

v7.1.0 · 2 years ago

90
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A simple, Twitter-aware tokenizer.

50Downloads across all versions in the last 7 days, from the official npm downloads API.CC-BY-NC-SA-3.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v7.1.0 (the latest release), from the OSV.dev database.

garu-minisearch-tokenizer

v0.4.1 · 1 month ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Korean tokenizer for MiniSearch, powered by garu-ko (1MB browser-native Korean morphological analyzer)

68Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.4.1 (the latest release), from the OSV.dev database.

marked-text-renderer

v0.1.0 · 12 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Renderer for marked outputting plain text that can easily be fed to a search indexer

114Downloads across all versions in the last 7 days, from the official npm downloads API.Apache v2License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.1.0 (the latest release), from the OSV.dev database.

@nexaloid/nexaloid

v0.1.34 · 1 month ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Chinese tokenizer runtime

74Downloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.1.34 (the latest release), from the OSV.dev database.

tokenizer-ui-kit

v0.3.21 · 3 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A Vue 3 UI Kit for Tokenizer - Complete component library

31Downloads across all versions in the last 7 days, from the official npm downloads API.ISCLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.3.21 (the latest release), from the OSV.dev database.

@prisma/orm-extension-paradedb

v8.0.0-rc.5 · 8 hours ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

ParadeDB BM25 full-text search indexes and operators for Prisma Next

4.2KDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v8.0.0-rc.5 (the latest release), from the OSV.dev database.

punkt

v1.0.2 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A port of NLTK's Punkt sentence tokenizer to JS.

299Downloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.2 (the latest release), from the OSV.dev database.

markdown-actions

v1.1.2 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A lightweight, zero-dependency JavaScript library for adding markdown formatting functionality to any textarea element

16.2KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.1.2 (the latest release), from the OSV.dev database.

yace

v1.1.0 · 1 month ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

tiny text editor with plugins

13.8KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.1.0 (the latest release), from the OSV.dev database.

ja-bigram-tokenizer

v0.1.2 · 11 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Character-bigram tokenizer for Japanese full-text search, with an Orama adapter

248Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.1.2 (the latest release), from the OSV.dev database.

ndx

v2.0.1 · 3 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Lightweight Full-Text Indexing and Searching Library.

614Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.0.1 (the latest release), from the OSV.dev database.

ora

v9.4.1 · 2 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Elegant terminal spinner

72.6MDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v9.4.1 (the latest release), from the OSV.dev database.

@lenml/tokenizer-gpt4o

v3.7.2 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

gpt4o tokenizer for NodeJS/Browser

159Downloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.7.2 (the latest release), from the OSV.dev database.

glsl-parser

v2.0.1 · 7 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

transform streamed glsl tokens into an ast

5.2KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.0.1 (the latest release), from the OSV.dev database.

monaco-ace-tokenizer

v0.2.3 · 3 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Syntax highlighting support for additional languages in monaco editor.

253Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.2.3 (the latest release), from the OSV.dev database.
Prev10 / 50Next