Discourse complements lexical semantics for non-factoid answer reranking

Peter Jansen, Mihai Surdeanu, Peter Clark

Research output: Chapter in Book/Report/Conference proceedingConference contribution

44 Citations (Scopus)

Abstract

We propose a robust answer reranking model for non-factoid questions that integrates lexical semantics with discourse information, driven by two representations of discourse: a shallow representation centered around discourse markers, and a deep one based on Rhetorical Structure Theory. We evaluate the proposed model on two corpora from different genres and domains: one from Yahoo! Answers and one from the biology domain, and two types of non-factoid questions: manner and reason. We experimentally demonstrate that the discourse structure of nonfactoid answers provides information that is complementary to lexical semantic similarity between question and answer, improving performance up to 24% (relative) over a state-of-the-art model that exploits lexical semantic similarity alone. We further demonstrate excellent domain transfer of discourse information, suggesting these discourse features have general utility to non-factoid question answering.

Original languageEnglish (US)
Title of host publication52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages977-986
Number of pages10
Volume1
ISBN (Print)9781937284725
StatePublished - 2014
Event52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Baltimore, MD, United States
Duration: Jun 22 2014Jun 27 2014

Other

Other52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014
CountryUnited States
CityBaltimore, MD
Period6/22/146/27/14

Fingerprint

semantics
discourse
Discourse
Lexical Semantics
biology
genre
performance
Semantic Similarity

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Jansen, P., Surdeanu, M., & Clark, P. (2014). Discourse complements lexical semantics for non-factoid answer reranking. In 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference (Vol. 1, pp. 977-986). Association for Computational Linguistics (ACL).

Discourse complements lexical semantics for non-factoid answer reranking. / Jansen, Peter; Surdeanu, Mihai; Clark, Peter.

52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference. Vol. 1 Association for Computational Linguistics (ACL), 2014. p. 977-986.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jansen, P, Surdeanu, M & Clark, P 2014, Discourse complements lexical semantics for non-factoid answer reranking. in 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference. vol. 1, Association for Computational Linguistics (ACL), pp. 977-986, 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, Baltimore, MD, United States, 6/22/14.
Jansen P, Surdeanu M, Clark P. Discourse complements lexical semantics for non-factoid answer reranking. In 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference. Vol. 1. Association for Computational Linguistics (ACL). 2014. p. 977-986
Jansen, Peter ; Surdeanu, Mihai ; Clark, Peter. / Discourse complements lexical semantics for non-factoid answer reranking. 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference. Vol. 1 Association for Computational Linguistics (ACL), 2014. pp. 977-986
@inproceedings{411eb03cacc44cf4ba955440bd4a1820,
title = "Discourse complements lexical semantics for non-factoid answer reranking",
abstract = "We propose a robust answer reranking model for non-factoid questions that integrates lexical semantics with discourse information, driven by two representations of discourse: a shallow representation centered around discourse markers, and a deep one based on Rhetorical Structure Theory. We evaluate the proposed model on two corpora from different genres and domains: one from Yahoo! Answers and one from the biology domain, and two types of non-factoid questions: manner and reason. We experimentally demonstrate that the discourse structure of nonfactoid answers provides information that is complementary to lexical semantic similarity between question and answer, improving performance up to 24{\%} (relative) over a state-of-the-art model that exploits lexical semantic similarity alone. We further demonstrate excellent domain transfer of discourse information, suggesting these discourse features have general utility to non-factoid question answering.",
author = "Peter Jansen and Mihai Surdeanu and Peter Clark",
year = "2014",
language = "English (US)",
isbn = "9781937284725",
volume = "1",
pages = "977--986",
booktitle = "52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference",
publisher = "Association for Computational Linguistics (ACL)",

}

TY - GEN

T1 - Discourse complements lexical semantics for non-factoid answer reranking

AU - Jansen, Peter

AU - Surdeanu, Mihai

AU - Clark, Peter

PY - 2014

Y1 - 2014

N2 - We propose a robust answer reranking model for non-factoid questions that integrates lexical semantics with discourse information, driven by two representations of discourse: a shallow representation centered around discourse markers, and a deep one based on Rhetorical Structure Theory. We evaluate the proposed model on two corpora from different genres and domains: one from Yahoo! Answers and one from the biology domain, and two types of non-factoid questions: manner and reason. We experimentally demonstrate that the discourse structure of nonfactoid answers provides information that is complementary to lexical semantic similarity between question and answer, improving performance up to 24% (relative) over a state-of-the-art model that exploits lexical semantic similarity alone. We further demonstrate excellent domain transfer of discourse information, suggesting these discourse features have general utility to non-factoid question answering.

AB - We propose a robust answer reranking model for non-factoid questions that integrates lexical semantics with discourse information, driven by two representations of discourse: a shallow representation centered around discourse markers, and a deep one based on Rhetorical Structure Theory. We evaluate the proposed model on two corpora from different genres and domains: one from Yahoo! Answers and one from the biology domain, and two types of non-factoid questions: manner and reason. We experimentally demonstrate that the discourse structure of nonfactoid answers provides information that is complementary to lexical semantic similarity between question and answer, improving performance up to 24% (relative) over a state-of-the-art model that exploits lexical semantic similarity alone. We further demonstrate excellent domain transfer of discourse information, suggesting these discourse features have general utility to non-factoid question answering.

UR - http://www.scopus.com/inward/record.url?scp=84906923809&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84906923809&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84906923809

SN - 9781937284725

VL - 1

SP - 977

EP - 986

BT - 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference

PB - Association for Computational Linguistics (ACL)

ER -