AI crawler blocking — generate robots.txt, meta tags, and HTTP headers for 30+ AI bots

The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s

set blocking stdio and stderr ensuring that terminal output does not truncate

A triple-linked lists based DOM implementation

WebDriver Proxy for testing rich clients. It block certain calls until Angular is done updating the page under test.

This repository contains a list of of HTTP user-agents used by robots, crawlers, and spiders as in single JSON file.

High Performance Layer 1 / Layer 2 Caching with Keyv Storage

Offload tasks to a pool of workers on node.js and in the browser

A library to recursively retrieve and serialize Notion pages with customization for machine learning applications.

Analyzes license information for multiple node.js modules (package.json files) as part of your software project.

This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

Server-side AI crawler tracking for DataFast

Crawler is a ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

Device detection module for Nuxt

Inspecting Node.js's Network with Chrome DevTools

Google Cloud Identity Platform Blocking Functions for Node.js

SEO & GEO audit CLI: 137 checks scoring how findable your site is by search engines and AI assistants (GPTBot, ClaudeBot, PerplexityBot). A-F grade, CI gate, SARIF/JUnit.

A polyfill for the proposed blocking elements stack API

Best-effort discovery of the machine's default gateway and local network IP exclusively with UDP sockets.