72,695 packages matching “website crawler”

crawl

v0.3.1 · 13 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Website crawler and differencer

22Downloads across all versions in the last 7 days, from the official npm downloads API.

No known vulnerabilities

Known vulnerabilities affecting v0.3.1 (the latest release), from the OSV.dev database.

super-simple-crawler

v1.2.4 · 9 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Super simple website crawler

16Downloads across all versions in the last 7 days, from the official npm downloads API.ISCLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.2.4 (the latest release), from the OSV.dev database.

fdir

v6.5.0 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s

168.8MDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v6.5.0 (the latest release), from the OSV.dev database.

sitebot

v2.2.0 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

SiteBot is an event driven website crawler.

8Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v2.2.0 (the latest release), from the OSV.dev database.

@charlietango/report

v1.0.9 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Display the result output from @charlietango/website-crawler

3Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.9 (the latest release), from the OSV.dev database.

@promptbook/website-crawler

v0.114.0-5 · 12 hours ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Promptbook: Create persistent AI agents that turn your company's scattered knowledge into action

1.5KDownloads across all versions in the last 7 days, from the official npm downloads API.BUSL-1.1License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.114.0-5 (the latest release), from the OSV.dev database.

@nelsonlaidev/scoutly

v0.4.0 · 4 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A fast, lightweight CLI website crawler and SEO analyzer built with Rust.

12Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.4.0 (the latest release), from the OSV.dev database.

js-website-crawler

v1.0.6 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Website crawler written in JavaScript built with Node.js.

3Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.6 (the latest release), from the OSV.dev database.

crawlio-mcp

v1.4.1 · 3 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

MCP server for Crawlio — control the macOS website crawler from AI assistants

6Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.4.1 (the latest release), from the OSV.dev database.

@charlietango/website-crawler

v0.0.14 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Crawl websites

7Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.0.14 (the latest release), from the OSV.dev database.

sitescrapr

v0.0.1 · 10 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Simple website crawler and scraper

2Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.0.1 (the latest release), from the OSV.dev database.

@datafast/ai-crawl

v1.0.9 · 21 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Server-side AI crawler tracking for DataFast

13.6KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.9 (the latest release), from the OSV.dev database.

dandy-crawl

v1.2.2 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A website crawler. Stays in the same website and return every internal linked URL with the associated HTTP status code

10Downloads across all versions in the last 7 days, from the official npm downloads API.GPL-3.0License declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.2.2 (the latest release), from the OSV.dev database.

linkedom

v0.18.13 · 28 days ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A triple-linked lists based DOM implementation

4.1MDownloads across all versions in the last 7 days, from the official npm downloads API.ISCLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.18.13 (the latest release), from the OSV.dev database.

prerender-node

v3.8.3 · 11 months ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

express middleware for serving prerendered javascript-rendered pages for SEO

21.4KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v3.8.3 (the latest release), from the OSV.dev database.

notion-md-crawler

v1.0.2 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A library to recursively retrieve and serialize Notion pages with customization for machine learning applications.

25.7KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.0.2 (the latest release), from the OSV.dev database.

crawler-user-agents

v1.56.0 · 1 month ago

100
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

This repository contains a list of of HTTP user-agents used by robots, crawlers, and spiders as in single JSON file.

55.4KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v1.56.0 (the latest release), from the OSV.dev database.

npm-license-crawler

v0.2.1 · 7 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

Analyzes license information for multiple node.js modules (package.json files) as part of your software project.

21.2KDownloads across all versions in the last 7 days, from the official npm downloads API.BSD-3-ClauseLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.2.1 (the latest release), from the OSV.dev database.

axe-crawler

v0.5.5 · 8 years ago

85
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

A highly configurable website crawler for automatically testing a website for accessibility issues using the axe-core library. Uses selenium and headless Chrome to load pages, inject axe-core, and run tests. Generates an html summary report in addition

20Downloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v0.5.5 (the latest release), from the OSV.dev database.

es6-crawler-detect

v4.0.2 · 1 year ago

95
0-100 score computed by this site from verifiable signals only: known OSV.dev advisories affecting this exact version, how recently it was published, and the repository's OpenSSF Scorecard.

This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

6.2KDownloads across all versions in the last 7 days, from the official npm downloads API.MITLicense declared in the package manifest.

No known vulnerabilities

Known vulnerabilities affecting v4.0.2 (the latest release), from the OSV.dev database.
1 / 50Next