TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems

Daniel Ferrés, Samir Kanaan, David Dominguez-Sal, Edgar González, Alicia Ageno, Maria Fuentes, Horacio Rodríguez, Mihai Surdeanu, Jordi Turmo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper describes the experiments of the TALPUPC group for factoid and 'other' (definitional) questions at TREC 2005 Main Question Answering (QA) task. Our current approach for factoid questions is based on a voting scheme among three QA systems: TALP-QA (our previous QA system), Sibyl (a new QA system developed at DAMA-UPC and TALP-UPC), and Aranea (a web-based data-driven approach). For defitional questions, we used two different systems: the TALP-QA Definitional system and LCSUM (a Summarization-based system). Our results for factoid questions indicate that the voting strategy improves the accuracy from 7.5% to 17.1%. While these numbers are low (due to technical problems in the Answer Extraction phase of TALPQA system) they indicate that voting is a succesful approach for performance boosting of QA systems. The answer to definitional questions is produced by selecting phrases using set of patterns associated with definitions. Its results are 17.2% of F-score in the best configuration of TALP-QA Definitional system.

Original languageEnglish (US)
Title of host publicationNIST Special Publication
StatePublished - 2005
Externally publishedYes
Event14th Text REtrieval Conference, TREC 2005 - Gaithersburg, MD, United States
Duration: Nov 15 2005Nov 18 2005

Other

Other14th Text REtrieval Conference, TREC 2005
CountryUnited States
CityGaithersburg, MD
Period11/15/0511/18/05

Fingerprint

Experiments

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Ferrés, D., Kanaan, S., Dominguez-Sal, D., González, E., Ageno, A., Fuentes, M., ... Turmo, J. (2005). TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems. In NIST Special Publication

TALP-UPC at TREC 2005 : Experiments using a voting scheme among three heterogeneous QA systems. / Ferrés, Daniel; Kanaan, Samir; Dominguez-Sal, David; González, Edgar; Ageno, Alicia; Fuentes, Maria; Rodríguez, Horacio; Surdeanu, Mihai; Turmo, Jordi.

NIST Special Publication. 2005.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ferrés, D, Kanaan, S, Dominguez-Sal, D, González, E, Ageno, A, Fuentes, M, Rodríguez, H, Surdeanu, M & Turmo, J 2005, TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems. in NIST Special Publication. 14th Text REtrieval Conference, TREC 2005, Gaithersburg, MD, United States, 11/15/05.
Ferrés D, Kanaan S, Dominguez-Sal D, González E, Ageno A, Fuentes M et al. TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems. In NIST Special Publication. 2005
Ferrés, Daniel ; Kanaan, Samir ; Dominguez-Sal, David ; González, Edgar ; Ageno, Alicia ; Fuentes, Maria ; Rodríguez, Horacio ; Surdeanu, Mihai ; Turmo, Jordi. / TALP-UPC at TREC 2005 : Experiments using a voting scheme among three heterogeneous QA systems. NIST Special Publication. 2005.
@inproceedings{7cca092070b0494ca981e9cb5c13bc3b,
title = "TALP-UPC at TREC 2005: Experiments using a voting scheme among three heterogeneous QA systems",
abstract = "This paper describes the experiments of the TALPUPC group for factoid and 'other' (definitional) questions at TREC 2005 Main Question Answering (QA) task. Our current approach for factoid questions is based on a voting scheme among three QA systems: TALP-QA (our previous QA system), Sibyl (a new QA system developed at DAMA-UPC and TALP-UPC), and Aranea (a web-based data-driven approach). For defitional questions, we used two different systems: the TALP-QA Definitional system and LCSUM (a Summarization-based system). Our results for factoid questions indicate that the voting strategy improves the accuracy from 7.5{\%} to 17.1{\%}. While these numbers are low (due to technical problems in the Answer Extraction phase of TALPQA system) they indicate that voting is a succesful approach for performance boosting of QA systems. The answer to definitional questions is produced by selecting phrases using set of patterns associated with definitions. Its results are 17.2{\%} of F-score in the best configuration of TALP-QA Definitional system.",
author = "Daniel Ferr{\'e}s and Samir Kanaan and David Dominguez-Sal and Edgar Gonz{\'a}lez and Alicia Ageno and Maria Fuentes and Horacio Rodr{\'i}guez and Mihai Surdeanu and Jordi Turmo",
year = "2005",
language = "English (US)",
booktitle = "NIST Special Publication",

}

TY - GEN

T1 - TALP-UPC at TREC 2005

T2 - Experiments using a voting scheme among three heterogeneous QA systems

AU - Ferrés, Daniel

AU - Kanaan, Samir

AU - Dominguez-Sal, David

AU - González, Edgar

AU - Ageno, Alicia

AU - Fuentes, Maria

AU - Rodríguez, Horacio

AU - Surdeanu, Mihai

AU - Turmo, Jordi

PY - 2005

Y1 - 2005

N2 - This paper describes the experiments of the TALPUPC group for factoid and 'other' (definitional) questions at TREC 2005 Main Question Answering (QA) task. Our current approach for factoid questions is based on a voting scheme among three QA systems: TALP-QA (our previous QA system), Sibyl (a new QA system developed at DAMA-UPC and TALP-UPC), and Aranea (a web-based data-driven approach). For defitional questions, we used two different systems: the TALP-QA Definitional system and LCSUM (a Summarization-based system). Our results for factoid questions indicate that the voting strategy improves the accuracy from 7.5% to 17.1%. While these numbers are low (due to technical problems in the Answer Extraction phase of TALPQA system) they indicate that voting is a succesful approach for performance boosting of QA systems. The answer to definitional questions is produced by selecting phrases using set of patterns associated with definitions. Its results are 17.2% of F-score in the best configuration of TALP-QA Definitional system.

AB - This paper describes the experiments of the TALPUPC group for factoid and 'other' (definitional) questions at TREC 2005 Main Question Answering (QA) task. Our current approach for factoid questions is based on a voting scheme among three QA systems: TALP-QA (our previous QA system), Sibyl (a new QA system developed at DAMA-UPC and TALP-UPC), and Aranea (a web-based data-driven approach). For defitional questions, we used two different systems: the TALP-QA Definitional system and LCSUM (a Summarization-based system). Our results for factoid questions indicate that the voting strategy improves the accuracy from 7.5% to 17.1%. While these numbers are low (due to technical problems in the Answer Extraction phase of TALPQA system) they indicate that voting is a succesful approach for performance boosting of QA systems. The answer to definitional questions is produced by selecting phrases using set of patterns associated with definitions. Its results are 17.2% of F-score in the best configuration of TALP-QA Definitional system.

UR - http://www.scopus.com/inward/record.url?scp=84873553732&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84873553732&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84873553732

BT - NIST Special Publication

ER -