Directory
Published directory entries matching your search.
A library support page explaining the invisible web, also called the deep web or hidden web. It states that search engines cover about 16% of the internet and advises against relying on them alone, especially for academic information. It lists content search engines cannot find, such as dynamically changing content, password protected resources and sites requiring registration, and suggests web directories and asking a librarian.
A 2001 book by Chris Sherman and Gary Price examining the nature and extent of the Invisible Web and offering pathfinders for accessing its content, aimed at novice and advanced web searchers. Chapters cover the development of the Internet, search engines and web directories, why search engines cannot see invisible web content, how to recognize it, and when to use it. A directory section lists more than 1,000 Invisible Web sites by topic.
The Overview of Medical Research in the Netherlands (OMON) is a website for searching medical research. It includes research from the former National Trial Register, ToetsingOnline, the Research Portal and the public portal of CTIS, and allows users to register their own research in the Research Portal. The former NTR information is available in English only, and other information is generally available in both Dutch and English.
Researchdata.se is a national web portal that allows users to find, share, and reuse research data from a wide range of research fields. It also provides guidance and recommendations for research data management. The portal is supported by several leading research infrastructures and works to advance open access to publicly funded research data.
Website.informer offers a free aggregated report on any domain, including WHOIS, visitors, IP and DNS details, competitors, and owners. It provides website statistics such as daily visitors, safety status, and global rank, plus competitor and similar-site research. It also offers IP and DNS lookup, WHOIS records, a browser extension for website analysis and security checks, and email-to-domain matching from WHOIS data.
Multi-agent Deep-research System is a Hugging Face Space created by Agents-MCP-Hackathon. The system uses AI agents to perform web searches and analyze information, generating detailed research reports in response to user-provided research questions. Users must supply API keys for web search functionality. The space is currently running.
Anara is an AI research assistant and workspace designed for research workflows. It allows users to search personal document libraries and the web, write with automatically generated source citations, and access answers that link to the exact source passage for verification. The tool is intended for high-stakes research use cases including academia.
Competitor Research is a competitive research tool. It takes a website URL and produces a ranked list of competitors, which users can curate into watchlists. It provides profiles covering positioning, pricing, launches, and other data, and sends weekly signals about competitor changes. Some features, including real-time change alerts and SEO and keyword movement, are listed as coming soon.
This arXiv paper by Dirk Lewandowski and Philipp Mayr provides a critical review of Bergman's 2001 study on the Deep Web. It introduces the concept of the Academic Invisible Web, defined as databases and collections relevant to academia but not searchable by general-purpose internet search engines, and reviews approaches for indexing it. The authors question Bergman's size estimate and suggest collaboration.
ESA Webb Telescope is the official European Space Agency X feed for the James Webb Space Telescope, an international partnership between NASA, ESA and CSA. It uses the hashtags #WebbSeesFarther and #BFFinSpace with @Hubble_Space.
An arXiv paper on discovering skylines over hidden web databases whose search interfaces enforce a top-k output constraint. It examines how search interface controls affect discovery cost and develops algorithms for one-ended range, free range and point predicates, combined for mixed interfaces. Experiments over real-world online and offline databases are reported.
An arXiv paper on digging deeper into hidden web databases limited by the top-k output constraint. It presents the GetNext meta-algorithm, which retrieves the next ranked tuple through a restrictive interface without prior knowledge of the ranking function, and algorithms that generate and execute multiple reformulated queries. Theoretical analysis and experiments over synthetic and real-world databases are reported.
National Academies chapter explaining deep-web content, discovery limits and information retrieval.
An online tool that parses the HTML of a web page and extracts its internal and external links, displaying them in plain text. After a page address is submitted, the system downloads the page and lists the resulting URLs. It is built with the Lynx command-line browser and described for research, development, security assessments and web page testing.
ImHex Web is a free and open source hex editor for reverse engineers, developers and data analysts. It runs directly in the web browser with the help of Emscripten and WebAssembly. A native version is also available.
OSINT Toolkit Website Information is a collection of tools for researching websites, IP addresses, domains, emails and MAC addresses. It lists lookups including Ping Website, Whois, reverse Whois, network and TLD lookup, IP history and geolocation, MAC vendor lookup, DNS report and header and source code search. The page states that some tools require the IP address, email address, domain name or MAC address of the target.
National Transportation Library record for research on discovering and cataloging hidden-web resources.
Raw Web helps research websites, public data, search results, or online communities.
WebCEO is an all-in-one SEO platform offering 24 online SEO tools for digital marketing, including keyword research, rankings, audits, backlink monitoring, link building and competitor research. It provides team collaboration, task management and branded SEO reports. A 14-day free trial is offered, and plans start from $99 per month.
Diigo is a research tool and knowledge-sharing community for collecting and tagging online resources, annotating web pages and PDFs, and organizing links and references. It offers highlights, sticky notes, an outliner, groups for sharing research, and archived web pages.
genei is a research tool that uses an AI algorithm to summarise articles, analyse research and find key information. It offers AI-powered summarisation and keyword extraction for groups of PDFs or webpages, document management with projects and folders, and reference generation. A Chrome extension add-on summarises webpages or saves them for later reading.
InfoLib is a financial research terminal combining options analytics, company research, SEC filings, institutional holdings, market data, screeners and calendars in one platform. It offers free tools without an account, plus paid plans: Options, Research and InfoLib Complete.
Research archive containing dark-web authorship verification datasets and baseline models.
GPT Researcher is an open deep research agent for web and local research. Its sources describe planner and execution agents that break tasks into research questions, with findings aggregated into cited research reports.
