github.com
Scrapling is an open source web scraping or crawling project for collecting website data.
Directory
Search results
Published directory entries matching your search.
github.com
Spider Suite is an open source web scraping or crawling project for collecting website data.
spidersuite.io
SpiderSuite helps crawl, scrape, copy, or extract data and pages from websites.
github.com
SuckIT is an open source project with code, documentation, and setup resources for web archiving and scraping.
tools.digitalmethods.net
Digital Methods tool for scraping search engine results and collecting web research data.
github.com
Trawl is an open source project with code, documentation, and setup resources for web archiving and scraping.
vegetableman.github.io
Vandal is a web archiving or scraping tool for saving pages, data, or online content.
machawk1.github.io
WAIL is a web archiving or scraping tool for saving pages, data, or online content.
github.com
Waymore is an open source project with code, documentation, and setup resources for web archiving and scraping.
webscrapeai.com
Webscrape AI is the perfect tool for collecting data from the web without the hassle of manual scraping. No coding skills required.
practicalbetterments.com
Wiki DL Guide is a web archiving or scraping tool for saving pages, data, or online content.
github.com
Awesome CSEs is an open source OSINT or search resource for online investigation workflows.
github.com
Awesome Free ChatGPT is a code project with source, releases, documentation, or setup notes.
pypi.org
Python OSINT tool for collecting public tweets, followers, likes, and profile data from Twitter.
alltheplaces.xyz
Open source data scraping software project.
crummy.com
Beautiful Soup: a library designed for screen-scraping HTML and XML.
github.com
Proxy service that helps scraping tools handle Cloudflare and browser-based anti-bot challenges.
github.com
A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.
github.com
Project Eyes On is a high-speed, multi-threaded surveillance tool by Y0oshi (@rde0) for locating open IP cameras worldwide. Unifies Google Dorking and Directory Scraping into a single OSINT engine.
github.com
Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, Coub, Vimeo, IFunny, VK, Odnoklassniki, Pikabu)
github.com
全自动短视频搬运工具，支持自动下载、去重、AI生成标题+标签、上传，可二开扩展至多平台，例如：TikTok->视频号/抖音/小红书、抖音->TikTok/视频号/小红书......video-processing, automation, tiktok, selenium, pyqt5, ffmpeg, bot, data-scraping, video-deduplication.
github.com
Curated list of awesome projects and resources related to Argo.
github.com
A curated list of awesome tools, demos, docs for ChatGPT and GPT-3, by @jordn.
prompts.chat
Awesome ChatGPT Prompts provides machine-learning models, research, training resources, or evaluation tools.