github.com
Awesome Web Scraping is an open source network utility for diagnostics, connectivity checks, or internet testing.
Directory
Search results
Published directory entries matching your search.
github.com
Dev Scraping APIs is a GitHub repository or organization with source code, releases, documentation, or project resources.
github.com
brozzler is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
datahoarder-website-to-markdown is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Dorks Eye Google Hacking Dork Scraping and Searching Script. Dorks Eye is a script I made in python 3. With this tool, you can easily find Google Dorks. Dork Eye collects potentially vulnerable web pages and applications on the Internet or other awesome info that is picked up by Google's search bots. Author: Jolanda de Koff
github.com
DownloadNet (dn) is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
grab-site is an open source web scraping or crawling project for collecting website data.
github.com
Heritrix3 is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Httrack is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
MarkSnip is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Resurrect Pages Fork is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Scoop is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Scrapling is an open source web scraping or crawling project for collecting website data.
github.com
Spider Suite is an open source web scraping or crawling project for collecting website data.
github.com
SuckIT is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Trawl is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Waymore is an open source project with code, documentation, and setup resources for web archiving and scraping.
github.com
Proxy service that helps scraping tools handle Cloudflare and browser-based anti-bot challenges.
github.com
A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.
github.com
Project Eyes On is a high-speed, multi-threaded surveillance tool by Y0oshi (@rde0) for locating open IP cameras worldwide. Unifies Google Dorking and Directory Scraping into a single OSINT engine.
github.com
Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, Coub, Vimeo, IFunny, VK, Odnoklassniki, Pikabu)
github.com
全自动短视频搬运工具，支持自动下载、去重、AI生成标题+标签、上传，可二开扩展至多平台，例如：TikTok->视频号/抖音/小红书、抖音->TikTok/视频号/小红书......video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.
github.com
specification.website is an open source tool or library for SEO, metadata, crawling, routing, or website optimization.
github.com
Alibi is a miscellaneous web resource for discovery, directories, archives, web culture, or useful tools.