tokengeex

ISCLicense declared in the package manifest.

This repository holds the code for the TokenGeeX Rust crate and Python package. TokenGeeX is a tokenizer for [CodeGeeX](https://github.com/THUDM/Codegeex2) aimed at code and Chinese. It is based on [UnigramLM (Taku Kudo 2018)](https://arxiv.org/abs/1804.1

NLPtokenizers
90

Security score

0 known advisories in v0.6.2

0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.advisory penalty −0 · staleness penalty −10

Weekly downloads

Downloads across all versions in the last 7 days, from the official npm downloads API (api.npmjs.org).

—

Unpacked size

Size of v0.6.2 on disk after npm install extracts the tarball, as reported by the npm registry.

2.8 MB

Dependencies

Direct runtime dependencies declared by v0.6.2. Transitive dependencies are not counted here — use the lockfile audit for the full tree.

0

Last publish

When v0.6.2 was published to the registry. Long gaps can indicate an unmaintained package.

2 years ago

Security advisories

Vulnerabilities affecting v0.6.2 specifically, from OSV.dev — the open database aggregating the GitHub Advisory Database (GHSA) and CVEs. Links open the full advisory.

No known vulnerabilities affect v0.6.2.