MetaSpider

Meta-searching and categorization on the Web

Hsinchun Chen, Haiyan Fan, Michael Chau, Dajun Zeng

Research output: Contribution to journalArticle

60 Citations (Scopus)

Abstract

It has become increasingly difficult to locate relevant information on the Web, even with the help of Web search engines. Two approaches to addressing the low precision and poor presentation of search results of current search tools are studied: meta-search and document categorization. Meta-search engines improve precision by selecting and integrating search results from generic or domain-specific Web search engines or other resources. Document categorization promises better organization and presentation of retrieved results. This article introduces MetaSpider, a meta-search engine that has real-time indexing and categorizing functions. We report in this paper the major components of MetaSpider and discuss related technical approaches. Initial results of a user evaluation study comparing MetaSpider, NorthernLight, and MetaCrawler in terms of clustering performance and of time and effort expended show that MetaSpider performed best in precision rate, but disclose no statistically significant differences in recall rate and time requirements. Our experimental study also reveals that MetaSpider exhibited a higher level of automation than the other two systems and facilitated efficient searching by providing the user with an organized, comprehensive view of the retrieved documents.

Original languageEnglish (US)
Pages (from-to)1134-1147
Number of pages14
JournalJournal of the American Society for Information Science and Technology
Volume52
Issue number13
DOIs
StatePublished - Nov 2001

Fingerprint

Search engines
search engine
indexing
automation
World Wide Web
Automation
organization
Search engine
evaluation
resources
performance
time
Metasearch
Web search

ASJC Scopus subject areas

  • Information Systems
  • Library and Information Sciences

Cite this

MetaSpider : Meta-searching and categorization on the Web. / Chen, Hsinchun; Fan, Haiyan; Chau, Michael; Zeng, Dajun.

In: Journal of the American Society for Information Science and Technology, Vol. 52, No. 13, 11.2001, p. 1134-1147.

Research output: Contribution to journalArticle

@article{68c6d792c3ff4e3ebe4c33834b70db2d,
title = "MetaSpider: Meta-searching and categorization on the Web",
abstract = "It has become increasingly difficult to locate relevant information on the Web, even with the help of Web search engines. Two approaches to addressing the low precision and poor presentation of search results of current search tools are studied: meta-search and document categorization. Meta-search engines improve precision by selecting and integrating search results from generic or domain-specific Web search engines or other resources. Document categorization promises better organization and presentation of retrieved results. This article introduces MetaSpider, a meta-search engine that has real-time indexing and categorizing functions. We report in this paper the major components of MetaSpider and discuss related technical approaches. Initial results of a user evaluation study comparing MetaSpider, NorthernLight, and MetaCrawler in terms of clustering performance and of time and effort expended show that MetaSpider performed best in precision rate, but disclose no statistically significant differences in recall rate and time requirements. Our experimental study also reveals that MetaSpider exhibited a higher level of automation than the other two systems and facilitated efficient searching by providing the user with an organized, comprehensive view of the retrieved documents.",
author = "Hsinchun Chen and Haiyan Fan and Michael Chau and Dajun Zeng",
year = "2001",
month = "11",
doi = "10.1002/asi.1180",
language = "English (US)",
volume = "52",
pages = "1134--1147",
journal = "Journal of the Association for Information Science and Technology",
issn = "2330-1635",
publisher = "John Wiley and Sons Ltd",
number = "13",

}

TY - JOUR

T1 - MetaSpider

T2 - Meta-searching and categorization on the Web

AU - Chen, Hsinchun

AU - Fan, Haiyan

AU - Chau, Michael

AU - Zeng, Dajun

PY - 2001/11

Y1 - 2001/11

N2 - It has become increasingly difficult to locate relevant information on the Web, even with the help of Web search engines. Two approaches to addressing the low precision and poor presentation of search results of current search tools are studied: meta-search and document categorization. Meta-search engines improve precision by selecting and integrating search results from generic or domain-specific Web search engines or other resources. Document categorization promises better organization and presentation of retrieved results. This article introduces MetaSpider, a meta-search engine that has real-time indexing and categorizing functions. We report in this paper the major components of MetaSpider and discuss related technical approaches. Initial results of a user evaluation study comparing MetaSpider, NorthernLight, and MetaCrawler in terms of clustering performance and of time and effort expended show that MetaSpider performed best in precision rate, but disclose no statistically significant differences in recall rate and time requirements. Our experimental study also reveals that MetaSpider exhibited a higher level of automation than the other two systems and facilitated efficient searching by providing the user with an organized, comprehensive view of the retrieved documents.

AB - It has become increasingly difficult to locate relevant information on the Web, even with the help of Web search engines. Two approaches to addressing the low precision and poor presentation of search results of current search tools are studied: meta-search and document categorization. Meta-search engines improve precision by selecting and integrating search results from generic or domain-specific Web search engines or other resources. Document categorization promises better organization and presentation of retrieved results. This article introduces MetaSpider, a meta-search engine that has real-time indexing and categorizing functions. We report in this paper the major components of MetaSpider and discuss related technical approaches. Initial results of a user evaluation study comparing MetaSpider, NorthernLight, and MetaCrawler in terms of clustering performance and of time and effort expended show that MetaSpider performed best in precision rate, but disclose no statistically significant differences in recall rate and time requirements. Our experimental study also reveals that MetaSpider exhibited a higher level of automation than the other two systems and facilitated efficient searching by providing the user with an organized, comprehensive view of the retrieved documents.

UR - http://www.scopus.com/inward/record.url?scp=0035498531&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0035498531&partnerID=8YFLogxK

U2 - 10.1002/asi.1180

DO - 10.1002/asi.1180

M3 - Article

VL - 52

SP - 1134

EP - 1147

JO - Journal of the Association for Information Science and Technology

JF - Journal of the Association for Information Science and Technology

SN - 2330-1635

IS - 13

ER -