Tom's Subject Directory & City Building Fan Site

Invisible (Deep) Web

The Invisible (Deep) Web Introduction

For years databases on the Web were referred to as specialty databases, subject-specific databases, virtual libraries, and other similar terms. The term "Invisible Web" was coined and these databases were referred to (and still are) as the invisible web. In the late 90's the term "deep Web" was coined and both invisible and deep web are used today.  [More]

The "Invisible Web" refers to Internet resources whose contents cannot or, more importantly, will not be indexed by search engines. These include databases and archived material, as well as general reference web sites. An increasing amount of deep Web content is opening up to free search as publishers and libraries make agreements with large search engines.

The Invisible Web consists of hundreds of trillions of web pages that are not indexed by the traditional search engines. The Invisible Web makes up a significant proportion of the web with an estimated size of between a half and two thirds of the total. Typical content of invisible web pages include interactive tools, archived material, statistical sources, news sources and a whole variety of full text databases such as dictionaries, encyclopedias and online books.

Invisible (Deep) Web Search Services

You may need more than Google and Wikipedia to get the job done. To find what you're looking for, it may be necessary to tap into the invisible web, the sites that don't get indexed by broad search engines. This page has a list of 99 resources that may help you.

"BrightPlanet has uncovered the "deep" Web a vast reservoir of Internet content that is 500 times larger than the known "surface" World Wide Web. The deep Web contains billions of high-quality documents in about 350,000 specialty databases -- all hidden from view from standard search engines."

INFOMINE was developed and is supported by Librarians from the University of California, Wake Forest University, California State University, the University of Detroit - Mercy and other universities and colleges have contributed to the building of INFOMINE.

INFOMINE is a virtual library of Internet resources relevant research at the university level. It contains useful Internet resources such as databases, electronic journals, electronic books, bulletin boards, mailing lists, online library card catalogs, articles, directories of researchers, and many other types of information.

The Institute of Museum and Library Services -IMLS - is the primary source of federal support for the nation’s 123,000 libraries and 17,500 museums.

Fund for the Improvement of Postsecondary Education - FIPSE - ia a part of ED.gov (U.S. Department of Education) which was created in 1980 by combining offices from several federal agencies.

The Librarians' Index to the Internet is a searchable, annotated subject directory of more than 7,000 internet resources selected and evaluated by librarians for their usefulness to users of public libraries. LII only includes links to the very best net content. While not a "pure" Invisible Web pathfinder, LII categorises each resource as best of, directories, databases, and specific resources. Databases, of course, are Invisible Web resources. By using LII's advanced search feature, you can limit your search to return only databases in the results list. Advanced search also lets you restrict your results to specific fields of the directory (author name, description, title, URL). The Librarians' Index to the Internet is a laser-sharp searching tool for finding Invisible Web databases.

"PubMed, a National Library of Medicine service, provides access to over 11 million MEDLINE citations back to the mid-1960's...including links to sites providing full text articles and other related resources."

Adobe and AltaVista teamed up to enable us to "search through more than a million summaries of Adobe Portable Document Format (PDF) files on the Web..." Now you can search through more than a million summaries of Adobe Portable Document Format (PDF) files on the Web. You can see the summaries in the search results before deciding to view the original Adobe PDF.

Searchable Databases

DeepPeep - database form search     [New Window]
DeepPeep is a search engine specialized in Web forms. The current beta version tracks 13,000 forms across 7 domains. The domains are Auto, Airfare, Biology, B ook, Hotel, Job and Rental. An advanced interface supports complex queries over the form repository (e.g., show the top-k labels in a domain, or top-k values for specific label in a domain. Advanced search allows you to perform more specific queries. Besides specifying keywords, you can also search for specific form element labels, i.e., the description of the form attributes.

Database for family-safe animal-related sites.

Searchable database of the Chicago Police Department's records of reported crime. You will be able to see maps, graphs, and tables of reported crime. The database contains 90 days of information which you can access in blocks of up to 14 days. Data is refreshed daily. However, the most recent information is back-dated 7 days from today's date.

Education Resources Information Center. An enormous database of education resources maintained by the United States federal Department of Education.

Provides access to statistics maintained by United States government agencies; most statistical data from the United States Census Bureau (and the equivalent bureau of other countries).

This database provides "authoritative conservation information on more than 50,000 plants, animals, and ecological communities" of the USA and Canada.

"Australian Geological Survey Organisation maintains a database of nuclear explosions with the location, time and size of explosions around the world since 1945."

Most people use this web site to get information about a particular number sequence. At the present time the table contains over 150000 sequences.

"Type in a series of numbers and this database will complete the sequence and provide the sequence name, along with its mathematical formula, structure, references, and links."

The Web version of the important Medline medical database.

Updates financial/currency rates once a minute.

Find the total population, the population density, and the other zip codes within the given radius of a particular zip code.

Invisible (Deep) Web Articles