kitoken

BSD-2-ClauseLicense declared in the package manifest.

Fast tokenizer for language models, supporting BPE, Unigram and WordPiece tokenization

tokenizernlpbpeunigramwordpiece
100

Security score

0 known advisories in v0.11.0

0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.advisory penalty −0 · staleness penalty −0

Weekly downloads

Downloads across all versions in the last 7 days, from the official npm downloads API (api.npmjs.org).

—

Unpacked size

Size of v0.11.0 on disk after npm install extracts the tarball, as reported by the npm registry.

7.5 MB

Dependencies

Direct runtime dependencies declared by v0.11.0. Transitive dependencies are not counted here — use the lockfile audit for the full tree.

0

Last publish

When v0.11.0 was published to the registry. Long gaps can indicate an unmaintained package.

2 months ago

Security advisories

Vulnerabilities affecting v0.11.0 specifically, from OSV.dev — the open database aggregating the GitHub Advisory Database (GHSA) and CVEs. Links open the full advisory.

No known vulnerabilities affect v0.11.0.