github.com
grab-site is a web crawler developed by ArchiveTeam for web archiving. It outputs WARC files, offers a dashboard for monitoring all crawls, and supports dynamic ignore patterns to customize crawling behavior.
Directory
Search results
Published directory entries matching your search.