2,533 packages matching “Crawl”
json-web-crawler
v0.8.3 · 4 years ago
Crawl website by json
No known vulnerabilities
unsplash-crawl
v0.0.2 · 11 years ago
Crawl all images from unsplash
No known vulnerabilities
@urdigital/mcp-server-bing-webmaster
v0.1.3 · 20 days ago
MCP server for Bing Webmaster Tools: sites, traffic/query/page stats, crawl issues, URL/sitemap submission, keyword stats.
No known vulnerabilities
crawl-readiness-mcp
v0.2.3 · 1 day ago
MCP server for Crawl Readiness — check AI readiness, validate schema/robots.txt, generate AI SEO files (llms.txt, robots.txt, JSON-LD), and read LLM Monitor share-of-voice trends directly from Claude, Cursor, and other MCP clients.
No known vulnerabilities
novada-mcp
v0.9.39 · 11 days ago
One MCP server for all web data. Search, scrape, crawl, proxy, and AI research in one install.
No known vulnerabilities
@profullstack/x402-gateway
v0.6.0 · 12 days ago
Sell crawl access to AI training crawlers by the day over x402, settled by CoinPay. One middleware: 402 with an offer, a sales page with CLI instructions, signed passes, and a robots.txt that keeps search crawlers welcome.
No known vulnerabilities
dsh-cloudflare-browser-run
v0.2.0 · 4 hours ago
DeepSeek Harness plugin: web browsing tools (markdown / screenshot / pdf / crawl) powered by Cloudflare Browser Run — real headless Chrome with JS rendering, login sessions and WebMCP support.
No known vulnerabilities
n8n-nodes-crawl-and-scrape2
v0.1.9 · 11 months ago
n8n custom node to crawl and scrape website
No known vulnerabilities
mcp-duckduckgo
v3.1.0 · 2 months ago
Free web search MCP server using DuckDuckGo — search, crawl, research, fetch, region/time/safe filters, Instant Answer, proxy auto-detect
No known vulnerabilities
pi-scraper
v0.13.1 · 2 months ago
Crawl, map, and structured extraction for Pi — scraper-first, Pi-native, and local-first.
No known vulnerabilities
@arachnodex/job-sitemap
v1.0.3 · 20 days ago
Arachnodex job for generating crawl sitemaps.
No known vulnerabilities
@nitpicker/report-google-sheets
v0.22.1 · 3 days ago
Nitpicker reporter that outputs crawl and analysis results to Google Sheets
No known vulnerabilities
sarvcrawl-cli
v1.0.1 · 11 days ago
Command-line interface for SarvCrawl — scrape, crawl, search, upload and manage knowledge bases from your terminal
No known vulnerabilities
@pymodel/dsh-tavily
v0.3.0 · 29 days ago
Tavily web search + Tavily MCP for DSH: settings toggle to official DeepSeek, keyless search, MCP extract/crawl/map
No known vulnerabilities
@viice/crawl-files
v1.0.7 · 6 years ago
crawl-files
No known vulnerabilities
crawl-fs
v0.0.1 · 10 years ago
Recursively crawl the content of a folder
No known vulnerabilities
@usedocent/cli
v0.2.0 · 7 days ago
Upload your app's routes and Markdown knowledge to Docent, or crawl a docs site into it.
No known vulnerabilities
@scrapio/mcp
v1.8.0 · 1 day ago
Official MCP server for the Scrapio — fetch, search, crawl, and interact from any AI agent
No known vulnerabilities
usetube
v2.2.7 · 3 years ago
crawl youtube without api key (search videos channels or get all channel/playlist's videos)
No known vulnerabilities
payload-plugin-rag-chatbot
v0.1.2 · 24 days ago
Self-hosted RAG chatbot plugin for Payload CMS 3 — crawl your site, answer visitor questions, no external vector store required.
No known vulnerabilities