25 packages matching “tokenise”
restore-redact
v1.0.1 · 2 months ago
Detect and tokenise secrets in objects/strings, then restore original values on demand
No known vulnerabilities
@genai-fi/nanogpt
v1.3.0 · 1 day ago
A browser-native implementation of small transformer language models using TensorFlow.js. This project is an educational toolkit for creating, training and running compact GPT-style models client-side. It supports model creation, tokenisation, dataset pre
No known vulnerabilities
dwc-gcode-core
v1.23.0 · 19 hours ago
RepRapFirmware-faithful G-code parsing for DuetWebControl plugins: an allocation-light line tokeniser, parameter parsing and in-place rewriting, conditional-G-code (if/var/while/...) recognition, a line-preserving config.g/macro directive editor, firmware
No known vulnerabilities
occam-grammars
v1.3.713 · 1 day ago
Occam's grammars.
No known vulnerabilities
architecture-mapper
v1.1.0 · 1 month ago
Abstract Syntax Tree analyser for visualising codebase architecture and dependencies
No known vulnerabilities
@utilitywarehouse/spreedly-tokenisation-component
v2.0.0 · 16 days ago
utilitywarehouse spreedly token
No known vulnerabilities
@unisim/doc
v0.7.0 · 23 days ago
The suite's document stack: read Word, OpenDocument, RTF, HTML, Markdown, CSV, JSON and text into one model, and write it back out as PDF, text, HTML, Markdown, CSV or JSON. No runtime dependencies.
No known vulnerabilities
@ccorgz/caiss
v1.1.1 · 7 days ago
Custom Adaptive Indexed Similarity Search — a fast, rule-driven, phonetic-aware in-memory search engine inspired by FAISS. Built for precision over raw vector similarity.
No known vulnerabilities
@shuttleglobal/twilio-demo-app
v1.0.23 · 1 month ago
Shuttle - Twilio Demo App
No known vulnerabilities
irctokens
v2.0.1 · 9 months ago
RFC1459 and IRCv3 protocol tokeniser
No known vulnerabilities
redacta-mcp
v2.1.0 · 1 month ago
MCP server that acts as a stateful privacy boundary: pseudonymises patient identifiers and PII before they reach an AI agent, keeps the reversal mapping outside the model context, and restores identifiers only at the trusted boundary. Runs locally, nothin
No known vulnerabilities
@ubcdigital/blocksnippets
v0.8.0 · 5 years ago
BlockSnippets is a source code snippets Library for UBC Digital Magazine
No known vulnerabilities
opencode-prompt-guard
v1.0.1 · 5 months ago
OpenCode plugin that scans tool outputs using Meta's Llama Prompt Guard 2
No known vulnerabilities
@emdzej/ncsx-fa-asw
v0.9.0 · 2 months ago
FA token string → ASW bit set converter. Spec: ../../docs/ecu-selection.md §3.3
No known vulnerabilities
stoke
v0.1.2 · 11 years ago
Generate the Abstract Syntax Tree (AST) of a Bash command.
No known vulnerabilities
@nvl/tree-sitter-sveltex
v0.3.0 · 3 months ago
Tree-sitter grammar for SvelTeX (.sveltex) documents
No known vulnerabilities
webidl2-mozilla-experimental
v1.0.8 · 7 years ago
This forked from https://github.com/w3c/webidl2.js
No known vulnerabilities
complexion-js
v0.2.2 · 5 years ago
Tokenize JavaScript
No known vulnerabilities
complexion
v0.2.2 · 5 years ago
A fast tokenizer/lexer core that is well tested.
No known vulnerabilities
xml-zero-lexer
v4.0.1 · 4 years ago
Friendly and forgiving HTML5/XML5/JSX lexer/parser with lots of tests. Memory-efficient and Web Worker compatible.
No known vulnerabilities