Tell me why: Using question answering as distant supervision for answer justification

Rebecca Sharp, Mihai Surdeanu, Peter Jansen, Marco A. Valenzuela-Escárcega, Peter Clark, Michael Hammond

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

For many applications of question answering (QA), being able to explain why a given model chose an answer is critical. However, the lack of labeled data for answer justifications makes learning this difficult and expensive. Here we propose an approach that uses answer ranking as distant supervision for learning how to select informative justifications, where justifications serve as inferential connections between the question and the correct answer while often containing little lexical overlap with either. We propose a neural network architecture for QA that reranks answer justifications as an intermediate (and human-interpretable) step in answer selection. Our approach is informed by a set of features designed to combine both learned representations and explicit features to capture the connection between questions, answers, and answer justifications. We show that with this end-to-end approach we are able to significantly improve upon a strong IR baseline in both justification ranking (+9% rated highly relevant) and answer selection (+6% P@1).

Original languageEnglish (US)
Title of host publicationCoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings
PublisherAssociation for Computational Linguistics (ACL)
Pages69-79
Number of pages11
ISBN (Electronic)9781945626548
StatePublished - Jan 1 2017
Event21st Conference on Computational Natural Language Learning, CoNLL 2017 - Vancouver, Canada
Duration: Aug 3 2017Aug 4 2017

Publication series

NameCoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings

Conference

Conference21st Conference on Computational Natural Language Learning, CoNLL 2017
CountryCanada
CityVancouver
Period8/3/178/4/17

Fingerprint

Network architecture
supervision
ranking
Neural networks
neural network
learning
lack

ASJC Scopus subject areas

  • Linguistics and Language
  • Artificial Intelligence
  • Human-Computer Interaction

Cite this

Sharp, R., Surdeanu, M., Jansen, P., Valenzuela-Escárcega, M. A., Clark, P., & Hammond, M. (2017). Tell me why: Using question answering as distant supervision for answer justification. In CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings (pp. 69-79). (CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings). Association for Computational Linguistics (ACL).

Tell me why : Using question answering as distant supervision for answer justification. / Sharp, Rebecca; Surdeanu, Mihai; Jansen, Peter; Valenzuela-Escárcega, Marco A.; Clark, Peter; Hammond, Michael.

CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings. Association for Computational Linguistics (ACL), 2017. p. 69-79 (CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sharp, R, Surdeanu, M, Jansen, P, Valenzuela-Escárcega, MA, Clark, P & Hammond, M 2017, Tell me why: Using question answering as distant supervision for answer justification. in CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings. CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings, Association for Computational Linguistics (ACL), pp. 69-79, 21st Conference on Computational Natural Language Learning, CoNLL 2017, Vancouver, Canada, 8/3/17.
Sharp R, Surdeanu M, Jansen P, Valenzuela-Escárcega MA, Clark P, Hammond M. Tell me why: Using question answering as distant supervision for answer justification. In CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings. Association for Computational Linguistics (ACL). 2017. p. 69-79. (CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings).
Sharp, Rebecca ; Surdeanu, Mihai ; Jansen, Peter ; Valenzuela-Escárcega, Marco A. ; Clark, Peter ; Hammond, Michael. / Tell me why : Using question answering as distant supervision for answer justification. CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings. Association for Computational Linguistics (ACL), 2017. pp. 69-79 (CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings).
@inproceedings{0f83d5a93922440a9edbdca8a493b91b,
title = "Tell me why: Using question answering as distant supervision for answer justification",
abstract = "For many applications of question answering (QA), being able to explain why a given model chose an answer is critical. However, the lack of labeled data for answer justifications makes learning this difficult and expensive. Here we propose an approach that uses answer ranking as distant supervision for learning how to select informative justifications, where justifications serve as inferential connections between the question and the correct answer while often containing little lexical overlap with either. We propose a neural network architecture for QA that reranks answer justifications as an intermediate (and human-interpretable) step in answer selection. Our approach is informed by a set of features designed to combine both learned representations and explicit features to capture the connection between questions, answers, and answer justifications. We show that with this end-to-end approach we are able to significantly improve upon a strong IR baseline in both justification ranking (+9{\%} rated highly relevant) and answer selection (+6{\%} P@1).",
author = "Rebecca Sharp and Mihai Surdeanu and Peter Jansen and Valenzuela-Esc{\'a}rcega, {Marco A.} and Peter Clark and Michael Hammond",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
series = "CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings",
publisher = "Association for Computational Linguistics (ACL)",
pages = "69--79",
booktitle = "CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings",

}

TY - GEN

T1 - Tell me why

T2 - Using question answering as distant supervision for answer justification

AU - Sharp, Rebecca

AU - Surdeanu, Mihai

AU - Jansen, Peter

AU - Valenzuela-Escárcega, Marco A.

AU - Clark, Peter

AU - Hammond, Michael

PY - 2017/1/1

Y1 - 2017/1/1

N2 - For many applications of question answering (QA), being able to explain why a given model chose an answer is critical. However, the lack of labeled data for answer justifications makes learning this difficult and expensive. Here we propose an approach that uses answer ranking as distant supervision for learning how to select informative justifications, where justifications serve as inferential connections between the question and the correct answer while often containing little lexical overlap with either. We propose a neural network architecture for QA that reranks answer justifications as an intermediate (and human-interpretable) step in answer selection. Our approach is informed by a set of features designed to combine both learned representations and explicit features to capture the connection between questions, answers, and answer justifications. We show that with this end-to-end approach we are able to significantly improve upon a strong IR baseline in both justification ranking (+9% rated highly relevant) and answer selection (+6% P@1).

AB - For many applications of question answering (QA), being able to explain why a given model chose an answer is critical. However, the lack of labeled data for answer justifications makes learning this difficult and expensive. Here we propose an approach that uses answer ranking as distant supervision for learning how to select informative justifications, where justifications serve as inferential connections between the question and the correct answer while often containing little lexical overlap with either. We propose a neural network architecture for QA that reranks answer justifications as an intermediate (and human-interpretable) step in answer selection. Our approach is informed by a set of features designed to combine both learned representations and explicit features to capture the connection between questions, answers, and answer justifications. We show that with this end-to-end approach we are able to significantly improve upon a strong IR baseline in both justification ranking (+9% rated highly relevant) and answer selection (+6% P@1).

UR - http://www.scopus.com/inward/record.url?scp=85051507685&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051507685&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85051507685

T3 - CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings

SP - 69

EP - 79

BT - CoNLL 2017 - 21st Conference on Computational Natural Language Learning, Proceedings

PB - Association for Computational Linguistics (ACL)

ER -