go to text version
Searchable Indexes

AltaVista
Excite
HotBot
Infoseek
Lycos
Northern Light
WebCrawler

Meta-searchers:
DogPile and Metafind
MetaCrawler
SavvySearch
search.com

Included here are selected services that provide searching of very large indexes of Internet resources. Each service is comprised of 3 components: a "robot" of some sort that automatically collects links, titles, and text from millions of Internet sites; a database where the resource information is stored; and a search engine that allows the user to interrogate the database for sites of interest. Some services also provide a limited browsable subject catalog, but the primary goal of each tool in this section is to provide a large, searchable database of Internet resources. In addition to these large indexes are meta-searchers, which take relatively simple input and search many indexes at once.

AltaVista
http://altavista.digital.com/
Provider: Digital Equipment Corporation
Browsable index: No
Search capabilities: Yes
Item submission method: Robot, also takes human submissions
Selection of Resources/Who Selects?: No
Annotated: Yes, from the page itself
Notable features: Size of the index, speed of retrieval
FAQ: Yes, for the product. Yes, for simple, and advanced search.
AltaVista indexes millions of Web pages as well as the text on those pages. It also provides access to thousands of Usenet newsgroups. Boolean AND/OR/NOT searching is supported, as well as phrase, proximity, truncation, and field searching. AltaVista provides feedback in order of relevance, but does not provide relevance scores. It has two detailed FAQs for searching that are essential to exploit the power of the system. It also allows users to refine their search via grouped tables of related terms for adding to or deleting from the search.
[Return to Top]

Excite
http://www.excite.com/
Provider: Excite, Inc.
Browsable index: Yes
Search capabilities: Yes
Item submission method: Robot, also takes human submissions
Selection of Resources/Who Selects?:No for search engine. Yes for browsable index/excite Editors
Annotated: Yes, from the page itself for the searchable index. Yes, by excite editors for the browsable index.
Notable feature: Size of index, concept based searching
FAQ: Yes, for the product. Yes, for the searching system
Excite indexes millions of Web pages and Usenet news articles. Its Intelligent Concept Extraction (ICE) search engine is based on concept searching, but also supports Boolean AND/OR/NOT, and phrase searching. Users may search by "concept" or key words. Concept based searching is easier and can yield larger retrieval, but can be confusing in that it sometimes returns sites that are not related to the query. Excite offers relevance feedback, returning different colored icons based on the level of relevance, and provides percentage confidence ratings. It allows the user to click on the relevance icon next to an item, to return similar items (query by example). It also offers the user the option of retrieval by site, with sites listed hierarchically. Users should read the searching FAQs carefully to use the system to its fullest advantage. Excite also contains a browsable subject index in 18 major categories.
[Return to Top]

HotBot
http://www.hotbot.com/
Provider: Inktomi and the HotWired Network
Browsable index: No
Search capabilities: Yes
Item submission method: Robot, also takes human submissions
Selection of Resources/Who Selects?: No
Annotated: Yes, from the page itself
Notable features: Size of the index, speed of retrieval
FAQ: Yes, for the product and the searching system
HotBot indexes millions of documents on the web and Usenet news. Through a forms based interface, it allows Boolean AND/OR/NOT and phrase searching. In "open all" mode, it also supports field searching. Items are returned in order of relevance, which is provided on a percentage basis.
[Return to Top]

Infoseek Ultraseek
http://www.infoseek.com/
Provider: Infoseek Corporation
Browsable index: Yes
Search capabilities: Yes
Item submission method: Robot, also takes human submissions
Selection of Resources/Who Selects?: No, for the search engine. Yes, for the browsable index/Infoseek editors
Annotated: Yes, from the page itself, for the search engine. Yes, by Infoseek editors, for the browsable index.
Notable feature: Size of index, speed of retrieval
FAQ: No, for the product. Yes, for the searching system.
Ultraseek indexes the index full text of millions of pages. It supports Boolean AND/OR/NOT and phrase searching, as well as field searching in four categories (link, site, url, and title). Items are returned in order of relevance, which is provided on a percentage basis. Infoseek also contains a browsable subject index in 18 major categories.
[Return to Top]

Lycos
http://www.lycos.com/
Provider: Lycos, Inc.
Browsable index: Yes.
Search capabilities: Yes
Item submission method: Robot, also takes human submissions
Selection of Resources/Who Selects?:No, for the search engine. Yes, for the two browsable indexes/Lycos editors.
Annotated: Yes, from the page itself, for the search engine. Yes, by Lycos editors, for the browsable indexes.
Notable feature: Size of index
FAQ: Yes, for the product and the searching system
Originally provided by Carnegie-Mellon University, this site is now maintained by Lycos, Inc. It supports Boolean AND/OR/NOT and truncation searching, relevance feedback, and allows the user to control the level and amount of feedback, as well as the level of relevance. It brings back annotation about the site from the page itself. Lycos also maintains two browsable subject indexes, the a2z Guide, and the well known Point Top 5% of all web sites directory.
[Return to Top]

Northern Light
http://www.northernlight.com/
Provider:Northern Light Technology LLC.
Browsable index: No.
Search capabilities: Yes
Item submission method: Robot, also takes human submissions
Selection of Resources/Who Selects?:No.
Annotated: Yes, from the page itself.
Notable feature: Organization of retrieval into subject based "custom search folders."
FAQ: Yes, for the product. Yes, for the searching system.
Northern Light adds a new wrinkle to Internet search indexes by way of its organization of retrieved results. Any search will retrieve "best documents" on the right side of the screen, along with a set of "custom search folders," (subject, type, source, or language) on the left side of the screen, to help users retrieve relevant items. Click on any of these folders, and a set of sub-folders will be opened. These folders are generated on the fly, rather than being part of a created hierarchy. Of course, the actual relevance of retrieved items for this, as for any large search index, depends on the ability of the searcher to narrow the search by use of Boolean tools. Also, as with any other large search index, there is no filtering for information quality. At present, Northern Light supports Boolean OR/NOT/MUST(+)/MUST NOT(-), phrase, truncation, and fielded searching. In addition to searching the web, users can search a database of information contained in journals, books, magazines, and newswires. Retrieval of these items is fee based.
[Return to Top]

WebCrawler
http://www.webcrawler.com/
Provider: Excite, Inc.
Browsable index: Yes
Search capabilities: Yes
Item submission method: Robot, also take human submissions
Selection of Resources/Who Selects?: No, for the search engine. Yes, for the browsable index/WebCrawler Eidtors.
Annotated: Yes, for the search engine, from the page itself. Yes, for the browsable index, by Webcrawler editors.
Notable feature: Quick and easy page locations
FAQ: Yes, for the product and the searching system
Originally provided by the University of Washington, then by America Online, this site is now maintained by Excite, Inc. It allows phrase, Boolean AND/OR/NOT, and proximity searching. Relevance feedback is available (if you select "Show Summaries"), as is a short summary taken from the page itself. It is a good basic searcher for a "quick and dirty" search. Webcrawler also contains a browsable subject index in 18 major categories.
[Return to Top]

Meta-searchers:

DogPile
http://www.dogpile.com/
and
MetaFind
http://www.metafind.com/
Provider: Aaron Flin and Unusual Solutions
Browsable index: No
Search capabilities: Yes
Item submission method: None
Selection of Resources/Who Selects?: No, however DogPile and MetaFind select the search indexes to be used.
Annotated: Yes, from the page itself
Notable features:Searches multiple indexes simultaneously
FAQ: Yes, for the product. Yes, for the searching system.
Don't be fooled by the frivolous attitude conveyed by Dogpile. Provided by Aaron Flin and Unusual Solutions, Dogpile is a powerful meta-search utility that will query up to 25 Web, Usenet, and FTP search indexes with one command. Boolean AND/NEAR/NOT and phrase searching is supported. Results are retrieved from three search engines at a time. The user controls which Internet protocols are searched and the time spent searching. In custom search mode, the user has complete control over both the indexes searched and the order searched. Unusual Solutions also provides MetaFind, which searches six web indexes, supports all the operators of Dogpile plus OR searching, and retrieves a limited number of "hits" from each index. Dogpile and MetaFind are fast and easy ways to query multiple search indexes, and together form a formidable individual effort.
[Return to Top]

MetaCrawler
http://www.metacrawler.com/
Provider: go2net, Inc.
Browsable index: No
Search capabilities: Yes
Item submission method: None
Selection of Resources/Who Selects?: No, however MetaCrawler selects the search indexes to be used.
Annotated: Yes, from the page itself
Notable features:Searches multiple indexes simultaneously
FAQ: Yes, for the product and the searching system.
With a single search request MetaCrawler searches six search engines: AltaVista, Excite, Infoseek, Lycos, Webcrawler, and Yahoo. It supports Boolean AND/OR and phrase searching. MetaCrawler collects confidence scores from each of the search engines used, combines them, and provides the search results in order of relevance based on the combined confidence score. It does not, however, return individual confidence scores. MetaCrawler allows the user to focus the search by geographic region and by selected Internet domain type, e.g. "com," "edu," and "gov." It also allows the user to specify search time spent and number of results per source returned.
[Return to Top]

SavvySearch
http://savvy.cs.colostate.edu:2000/
Provider: Daniel Dreilinger
Browsable index: No
Search capabilities: Yes
Item submission method: None
Selection of Resources/Who Selects?: No, however SavvySearch selects the search indexes to be used
Annotated: Yes, from the page itself
Notable features: Queries over 20 search engines with one command. Groups results by sets of search engines. Context sensitive help is available. Query form is available in multiple languages.
FAQ:Yes, for the product. Yes, for the searching system. Context sensitive help is available by clicking on any blue box with an "i" in it.
SavvySearch allows the user to enter a single query to search over 20 search engines. (The list of engines can be found on the SavvySearch front page.) Search results can include Web sites, software, email addresses, and even movies. Boolean AND/OR and phrase searching are supported.Results are returned in a "search plan," with the best matches and their constituent search engines listed first. Alternatively, the user can request that the results be integrated into a single list of references, in which case duplicates are deleted and the constituent search engines are not specified. The user has some control of how many hits are gathered from each engine, as well as the amount of information displayed. The short searching FAQ should be read carefully in order to fully exploit the system
[Return to Top]

search.com
http://www.search.com
Provider: c|net, inc.
Browsable index: Yes, of searching engines
Search capabilities: Yes
Item submission method: None
Selection of Resources/Who Selects?: No, however search.com selects the search indexes to be used
Annotated: Yes, by search.com editors
Notable features: The number of search engines available. User can customize a page of favorite search engines. Brief annotations and searching tips for each search engine are provided.
FAQ:Yes, for the product. Yes, for the searching system. A Boolean searching primer is also provided.
Search.com provides the user with direct access to hundreds of search engines. The engines are organized into over 25 subject categories, allowing the user to narrow a search by selecting engines specializing in general topics such as art, science, health, news, sports, or entertainment. Each engine is accompanied by a short annotation, as well as one or two searching tips for that engine. The best engines, as determined by search.com, are indicated by a "top pick" icon. The search.com service allows the user to create a personalized page of useful engines that will appear for that user each time search.com initializes. Each subject heirarchy can be searched across selected search engines. The number of engines available, as well as the organization of the site, make it a valuable addition as an Internet searching tool.
[Return to Top]