This arXiv paper proposes an end-to-end scalable architecture for early identification of new Tor sites and daily analysis of their content. It is built on an open source big data stack including Kubernetes, Kafka, Kubeflow and MinIO, discovering onion addresses across several sources, downloading HTML from Tor, deduplicating with MinHash LSH and categorizing with BERTopic modeling. In 93 days it identified 80,049 onion services and characterized 90% of them.
Dizzy is an open-source crawling and analysis system for onion services. It implements techniques to explore, update, check, and classify onion services at scale without overwhelming the Tor network. It was deployed in April 2021 and used to analyze more than 63.3 million crawled onion webpages, focusing on domain operations, web content, cryptocurrency usage, and web graph.
The Guardian SecureDrop is a tool for securely sending messages and documents to the Guardian. It uses the Tor network to hide where messages came from, and communication is encrypted in transit and on the server. Sources access it through an onion URL using the Tor browser.
News24 offers ways to send confidential news tips to its journalists, including email and SecureDrop, which encrypts submissions and stores them on a physically secured computer with decryption on an offline machine. The page gives instructions for using SecureDrop through the Tor browser and lists traditional mail addresses.
Public-health research measuring harmful-content availability and searches on the Tor network.
CRATOR is a proposed dark web crawler described in an arXiv paper. It is designed to extract pages that handle security protocols such as captchas, using seed URL lists, link analysis and scanning to discover content, with user-agent rotation and proxy usage for anonymity. The authors report evaluation using coverage, performance and robustness metrics, and suggest threat intelligence, cybersecurity and online investigation applications.
An arXiv paper measuring onion website discovery and Tor users' interests with honeypots. The authors deployed honeypot onion websites in March–April 2025 and seeded links via three channels to observe interaction events. They report that human users almost without exception originated from Ahmia.fi, and that the CSAM-themed honeypot drew markedly higher engagement than the others.
Community notes
Be the first to share how this project worked for you.
Sign in with a verified account to leave a review or join the discussion.
Sign in to reviewKeep exploring
Dizzy is an open-source crawling and analysis system for onion services. It implements techniques to explore, update, check, and classify onion services at scale without overwhelming the Tor network. It was deployed in April 2021 and used to analyze more than 63.3 million crawled onion webpages, focusing on domain operations, web content, cryptocurrency usage, and web graph.
The Guardian SecureDrop is a tool for securely sending messages and documents to the Guardian. It uses the Tor network to hide where messages came from, and communication is encrypted in transit and on the server. Sources access it through an onion URL using the Tor browser.
News24 offers ways to send confidential news tips to its journalists, including email and SecureDrop, which encrypts submissions and stores them on a physically secured computer with decryption on an offline machine. The page gives instructions for using SecureDrop through the Tor browser and lists traditional mail addresses.
Public-health research measuring harmful-content availability and searches on the Tor network.
CRATOR is a proposed dark web crawler described in an arXiv paper. It is designed to extract pages that handle security protocols such as captchas, using seed URL lists, link analysis and scanning to discover content, with user-agent rotation and proxy usage for anonymity. The authors report evaluation using coverage, performance and robustness metrics, and suggest threat intelligence, cybersecurity and online investigation applications.
An arXiv paper measuring onion website discovery and Tor users' interests with honeypots. The authors deployed honeypot onion websites in March–April 2025 and seeded links via three channels to observe interaction events. They report that human users almost without exception originated from Ahmia.fi, and that the CSAM-themed honeypot drew markedly higher engagement than the others.
