Search results

Dev Scraping APIs

github.com

Dev Scraping APIs is a GitHub repository or organization with source code, releases, documentation, or project resources.

Awesome Web Scraping

github.com

Awesome Web Scraping is an open source network utility for diagnostics, connectivity checks, or internet testing.

piko (Twitter)

github.com

piko (Twitter) is a code project with source, releases, documentation, or setup notes.

Twitter Archive Parser is a code project with source, releases, documentation, or setup notes.

Twitter Detector is a code project with source, releases, documentation, or setup notes.

Twitter Mass Unfollow is a code project with source, releases, documentation, or setup notes.

twitter-web-exporter is a code project with source, releases, documentation, or setup notes.

TwitterMediaHarvest is a code project with source, releases, documentation, or setup notes.

Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, Coub, Vimeo, IFunny, VK, Odnoklassniki, Pikabu)

Synchronizes posts from Mastodon to Twitter and back

Twitter Text Java

github.com

A Java implementation of Twitter's text processing library.

brozzler

github.com

brozzler is an open source project with code, documentation, and setup resources for web archiving and scraping.

datahoarder-website-to-markdown

github.com

datahoarder-website-to-markdown is an open source project with code, documentation, and setup resources for web archiving and scraping.

Dorks Eye Google Hacking Dork Scraping and Searching Script. Dorks Eye is a script I made in python 3. With this tool, you can easily find Google Dorks. Dork Eye collects potentially vulnerable web pages and applications on the Internet or other awesome info that is picked up by Google's search bots. Author: Jolanda de Koff

DownloadNet (dn)

github.com

DownloadNet (dn) is an open source project with code, documentation, and setup resources for web archiving and scraping.

FlareSolverr

github.com

Proxy service that helps scraping tools handle Cloudflare and browser-based anti-bot challenges.

A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.

grab-site

github.com

grab-site is an open source web scraping or crawling project for collecting website data.

Heritrix3

github.com

Heritrix3 is an open source project with code, documentation, and setup resources for web archiving and scraping.

Httrack

github.com

Httrack is an open source project with code, documentation, and setup resources for web archiving and scraping.

MarkSnip is an open source project with code, documentation, and setup resources for web archiving and scraping.

Project Eyes On is a high-speed, multi-threaded surveillance tool by Y0oshi (@rde0) for locating open IP cameras worldwide. Unifies Google Dorking and Directory Scraping into a single OSINT engine.

Resurrect Pages Fork

github.com

Resurrect Pages Fork is an open source project with code, documentation, and setup resources for web archiving and scraping.

Scoop

github.com

Scoop is an open source project with code, documentation, and setup resources for web archiving and scraping.