16,948 packages matching “Sentence tokenizer”

glsl-token-properties

v1.0.1 · 11 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Takes an array of GLSL tokens and determines whether or not they're a property of another identifier

784.8KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.1 (the latest release), from the OSV.dev database.

gpt3-tokenizer

v1.1.5 · 3 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

[![Build](https://github.com/botisan-ai/gpt3-tokenizer/actions/workflows/main.yml/badge.svg)](https://github.com/botisan-ai/gpt3-tokenizer/actions/workflows/main.yml) [![NPM Version](https://img.shields.io/npm/v/gpt3-tokenizer.svg)](https://www.npmjs.com/

22.3KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.1.5 (the latest release), from the OSV.dev database.

@adguard/css-tokenizer

v1.2.0 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

CSS / Extended CSS tokenizer

5.5KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.2.0 (the latest release), from the OSV.dev database.

glsl-token-whitespace-trim

v1.0.0 · 10 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Trim the whitespace within an array of GLSL tokens

783.8KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.0 (the latest release), from the OSV.dev database.

@csstools/css-color-parser

v4.2.0 · 7 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Parse CSS color values

55.0MDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v4.2.0 (the latest release), from the OSV.dev database.

@yozora/tokenizer-inline-math

v2.4.0 · 21 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Tokenizer for processing inline math (formulas)

8.2KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.4.0 (the latest release), from the OSV.dev database.

code-error-fragment

v0.0.230 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Shows code error fragment of input file

571.4KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.0.230 (the latest release), from the OSV.dev database.

@microsoft/tiktokenizer

v1.0.10 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Tokenizer for OpenAI large language models.

79.2KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.10 (the latest release), from the OSV.dev database.

@huggingface/tokenizers

v0.1.3 · 5 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

🤗 Tokenizers.js: A pure JS/TS implementation of today's most used tokenizers

1.3MDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.1.3 (the latest release), from the OSV.dev database.

@yozora/tokenizer-admonition

v2.4.0 · 21 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Tokenizer for processing admonitions

9.7KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.4.0 (the latest release), from the OSV.dev database.

@connectrpc/connect-node

v2.1.2 · 2 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Connect is a family of libraries for building and consuming APIs on different languages and platforms, and [@connectrpc/connect](https://www.npmjs.com/package/@connectrpc/connect) brings type-safe APIs with Protobuf to TypeScript.

1.9MDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.1.2 (the latest release), from the OSV.dev database.

@lenml/tokenizers

v3.7.2 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

a lightweight no-dependency fork of transformers.js (only tokenizers)

55.6KDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.7.2 (the latest release), from the OSV.dev database.

zeparser

v0.0.7 · 13 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

My JavaScript parser

44.8KDownloads across all versions in the last 7 days, from the official npm downloads API.

No known vulnerabilities

Known vulnerabilities affecting v0.0.7 (the latest release), from the OSV.dev database.

@connectrpc/connect-web

v2.1.2 · 2 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Connect is a family of libraries for building and consuming APIs on different languages and platforms. [@connectrpc/connect](https://www.npmjs.com/package/@connectrpc/connect) brings type-safe APIs with Protobuf to TypeScript.

2.5MDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.1.2 (the latest release), from the OSV.dev database.

llama3-tokenizer-js

v1.2.0 · 2 years ago

90
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

JS tokenizer for LLaMA 3

2.9KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.2.0 (the latest release), from the OSV.dev database.

title-case

v4.3.2 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Transform a string into title case following English rules

10.4MDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v4.3.2 (the latest release), from the OSV.dev database.

@lenml/tokenizer-claude

v3.7.2 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

claude tokenizer for NodeJS/Browser

5.8KDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.7.2 (the latest release), from the OSV.dev database.

@lenml/tokenizer-gemini

v3.7.2 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

gemini tokenizer for NodeJS/Browser

5.1KDownloads across all versions in the last 7 days, from the official npm downloads API.Apache-2.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.7.2 (the latest release), from the OSV.dev database.

vue-eslint-parser-template-tokenizer-pug

v1.0.0 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

This is an internal library. If you just want to lint your pug templates, use [eslint-plugin-vue-pug](https://github.com/rashfael/eslint-plugin-vue-pug).

24.2KDownloads across all versions in the last 7 days, from the official npm downloads API.ISCLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.0 (the latest release), from the OSV.dev database.

@yozora/core-tokenizer

v2.4.0 · 21 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

<header> <h1 align="center"> <a href="https://github.com/yozorajs/yozora/tree/v2.3.17/packages/core-tokenizer#readme">@yozora/core-tokenizer</a> </h1> <div align="center"> <a href="https://www.npmjs.com/package/@yozora/core-tokenizer">

10.3KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.4.0 (the latest release), from the OSV.dev database.
Prev3 / 50Next