Performance analysis of a distributed question/answering system

Mihai Surdeanu, Dan I. Moldovan, Sanda M. Harabagiu

Research output: Contribution to journalArticle

29 Citations (Scopus)

Abstract

The problem of question/answering (Q/A) is to find answers to open-domain questions by searching large collections of documents. Unlike information retrieval systems very common today in the form of Internet search engines, Q/A systems do not retrieve documents, but instead provide short, relevant answers located in small fragments of text. This enhanced functionality comes with a price: Q/A systems are significantly slower and require more hardware resources than information retrieval systems. This paper proposes a distributed Q/A architecture that enhances the system throughput through the exploitation of interquestion parallelism and dynamic load balancing and reduces the individual question response time through the exploitation of intraquestion parallelism. Inter and intraquestion parallelism are both exploited using several scheduling points: one before the Q/A task is started and two embedded in the Q/A task. An analytical performance model is introduced. The model analyzes both the interquestion parallelism overhead generated by the migration of questions and the intraquestion parallelism overhead generated by the partitioning of the Q/A task. The analytical model indicates that both question migration and partitioning are required for a high-performance system: Intraquestion parallelism leads to significant speedup of individual questions, but it is practical up to about 90 processors, depending on the system parameters. The exploitation of intertask parallelism provides a scalable way to improve the system throughput. The distributed Q/A system has been implemented on a network of 16 Pentium III computers. The experimental results indicate that, at high system load, the dynamic load balancing strategy proposed in this paper outperforms two other traditional approaches. At low system load, the distributed Q/A system reduces question response times through task partitioning, with factors close to the ones indicated by the analytical model.

Original languageEnglish (US)
Pages (from-to)579-596
Number of pages18
JournalIEEE Transactions on Parallel and Distributed Systems
Volume13
Issue number6
DOIs
StatePublished - Jun 2002
Externally publishedYes

Fingerprint

Question Answering System
Information retrieval systems
Dynamic loads
Resource allocation
Performance Analysis
Distributed Systems
Analytical models
Parallelism
Throughput
Question Answering
Search engines
Computer hardware
Loads (forces)
Computer systems
Scheduling
Internet
Exploitation
Analytical Model
Partitioning
Dynamic Load Balancing

Keywords

  • Distributed question answering
  • Load balancing
  • Migration
  • Partitioning

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Theoretical Computer Science
  • Computational Theory and Mathematics

Cite this

Performance analysis of a distributed question/answering system. / Surdeanu, Mihai; Moldovan, Dan I.; Harabagiu, Sanda M.

In: IEEE Transactions on Parallel and Distributed Systems, Vol. 13, No. 6, 06.2002, p. 579-596.

Research output: Contribution to journalArticle

@article{9506e883c3cd497da09a8b79f27223ad,
title = "Performance analysis of a distributed question/answering system",
abstract = "The problem of question/answering (Q/A) is to find answers to open-domain questions by searching large collections of documents. Unlike information retrieval systems very common today in the form of Internet search engines, Q/A systems do not retrieve documents, but instead provide short, relevant answers located in small fragments of text. This enhanced functionality comes with a price: Q/A systems are significantly slower and require more hardware resources than information retrieval systems. This paper proposes a distributed Q/A architecture that enhances the system throughput through the exploitation of interquestion parallelism and dynamic load balancing and reduces the individual question response time through the exploitation of intraquestion parallelism. Inter and intraquestion parallelism are both exploited using several scheduling points: one before the Q/A task is started and two embedded in the Q/A task. An analytical performance model is introduced. The model analyzes both the interquestion parallelism overhead generated by the migration of questions and the intraquestion parallelism overhead generated by the partitioning of the Q/A task. The analytical model indicates that both question migration and partitioning are required for a high-performance system: Intraquestion parallelism leads to significant speedup of individual questions, but it is practical up to about 90 processors, depending on the system parameters. The exploitation of intertask parallelism provides a scalable way to improve the system throughput. The distributed Q/A system has been implemented on a network of 16 Pentium III computers. The experimental results indicate that, at high system load, the dynamic load balancing strategy proposed in this paper outperforms two other traditional approaches. At low system load, the distributed Q/A system reduces question response times through task partitioning, with factors close to the ones indicated by the analytical model.",
keywords = "Distributed question answering, Load balancing, Migration, Partitioning",
author = "Mihai Surdeanu and Moldovan, {Dan I.} and Harabagiu, {Sanda M.}",
year = "2002",
month = "6",
doi = "10.1109/TPDS.2002.1011413",
language = "English (US)",
volume = "13",
pages = "579--596",
journal = "IEEE Transactions on Parallel and Distributed Systems",
issn = "1045-9219",
publisher = "IEEE Computer Society",
number = "6",

}

TY - JOUR

T1 - Performance analysis of a distributed question/answering system

AU - Surdeanu, Mihai

AU - Moldovan, Dan I.

AU - Harabagiu, Sanda M.

PY - 2002/6

Y1 - 2002/6

N2 - The problem of question/answering (Q/A) is to find answers to open-domain questions by searching large collections of documents. Unlike information retrieval systems very common today in the form of Internet search engines, Q/A systems do not retrieve documents, but instead provide short, relevant answers located in small fragments of text. This enhanced functionality comes with a price: Q/A systems are significantly slower and require more hardware resources than information retrieval systems. This paper proposes a distributed Q/A architecture that enhances the system throughput through the exploitation of interquestion parallelism and dynamic load balancing and reduces the individual question response time through the exploitation of intraquestion parallelism. Inter and intraquestion parallelism are both exploited using several scheduling points: one before the Q/A task is started and two embedded in the Q/A task. An analytical performance model is introduced. The model analyzes both the interquestion parallelism overhead generated by the migration of questions and the intraquestion parallelism overhead generated by the partitioning of the Q/A task. The analytical model indicates that both question migration and partitioning are required for a high-performance system: Intraquestion parallelism leads to significant speedup of individual questions, but it is practical up to about 90 processors, depending on the system parameters. The exploitation of intertask parallelism provides a scalable way to improve the system throughput. The distributed Q/A system has been implemented on a network of 16 Pentium III computers. The experimental results indicate that, at high system load, the dynamic load balancing strategy proposed in this paper outperforms two other traditional approaches. At low system load, the distributed Q/A system reduces question response times through task partitioning, with factors close to the ones indicated by the analytical model.

AB - The problem of question/answering (Q/A) is to find answers to open-domain questions by searching large collections of documents. Unlike information retrieval systems very common today in the form of Internet search engines, Q/A systems do not retrieve documents, but instead provide short, relevant answers located in small fragments of text. This enhanced functionality comes with a price: Q/A systems are significantly slower and require more hardware resources than information retrieval systems. This paper proposes a distributed Q/A architecture that enhances the system throughput through the exploitation of interquestion parallelism and dynamic load balancing and reduces the individual question response time through the exploitation of intraquestion parallelism. Inter and intraquestion parallelism are both exploited using several scheduling points: one before the Q/A task is started and two embedded in the Q/A task. An analytical performance model is introduced. The model analyzes both the interquestion parallelism overhead generated by the migration of questions and the intraquestion parallelism overhead generated by the partitioning of the Q/A task. The analytical model indicates that both question migration and partitioning are required for a high-performance system: Intraquestion parallelism leads to significant speedup of individual questions, but it is practical up to about 90 processors, depending on the system parameters. The exploitation of intertask parallelism provides a scalable way to improve the system throughput. The distributed Q/A system has been implemented on a network of 16 Pentium III computers. The experimental results indicate that, at high system load, the dynamic load balancing strategy proposed in this paper outperforms two other traditional approaches. At low system load, the distributed Q/A system reduces question response times through task partitioning, with factors close to the ones indicated by the analytical model.

KW - Distributed question answering

KW - Load balancing

KW - Migration

KW - Partitioning

UR - http://www.scopus.com/inward/record.url?scp=0036612649&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036612649&partnerID=8YFLogxK

U2 - 10.1109/TPDS.2002.1011413

DO - 10.1109/TPDS.2002.1011413

M3 - Article

AN - SCOPUS:0036612649

VL - 13

SP - 579

EP - 596

JO - IEEE Transactions on Parallel and Distributed Systems

JF - IEEE Transactions on Parallel and Distributed Systems

SN - 1045-9219

IS - 6

ER -