TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems

Daniel Ferrés, Samir Kanaan, David Dominguez-Sal, Edgar González, Alicia Ageno, Maria Fuentes, Horacio Rodríguez, Mihai Surdeanu, Jordi Turmo

Research output: Contribution to journalConference article

Abstract

This paper describes the experiments of the TALPUPC group for factoid and 'other' (definitional) questions at TREC 2005 Main Question Answering (QA) task. Our current approach for factoid questions is based on a voting scheme among three QA systems: TALP-QA (our previous QA system), Sibyl (a new QA system developed at DAMA-UPC and TALP-UPC), and Aranea (a web-based data-driven approach). For defitional questions, we used two different systems: the TALP-QA Definitional system and LCSUM (a Summarization-based system). Our results for factoid questions indicate that the voting strategy improves the accuracy from 7.5% to 17.1%. While these numbers are low (due to technical problems in the Answer Extraction phase of TALPQA system) they indicate that voting is a succesful approach for performance boosting of QA systems. The answer to definitional questions is produced by selecting phrases using set of patterns associated with definitions. Its results are 17.2% of F-score in the best configuration of TALP-QA Definitional system.

Original languageEnglish (US)
JournalNIST Special Publication
StatePublished - Dec 1 2005
Externally publishedYes
Event14th Text REtrieval Conference, TREC 2005 - Gaithersburg, MD, United States
Duration: Nov 15 2005Nov 18 2005

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems'. Together they form a unique fingerprint.

  • Cite this

    Ferrés, D., Kanaan, S., Dominguez-Sal, D., González, E., Ageno, A., Fuentes, M., Rodríguez, H., Surdeanu, M., & Turmo, J. (2005). TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems. NIST Special Publication.