TextGraphs 2019 shared task on multi-hop inference for explanation regeneration

Peter Jansen, Dmitry Ustalov

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

While automated question answering systems are increasingly able to retrieve answers to natural language questions, their ability to generate detailed human-readable explanations for their answers is still quite limited. The Shared Task on Multi-Hop Inference for Explanation Regeneration tasks participants with regenerating detailed gold explanations for standardized elementary science exam questions by selecting facts from a knowledge base of semistructured tables. Each explanation contains between 1 and 16 interconnected facts that form an "explanation graph" spanning core scientific knowledge and detailed world knowledge. It is expected that successfully combining these facts to generate detailed explanations will require advancing methods in multihop inference and information combination, and will make use of the supervised training data provided by the WorldTree explanation corpus. The top-performing system achieved a mean average precision (MAP) of 0.56, substantially advancing the state-of-the-art over a baseline information retrieval model. Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low, highlighting the difficulty of generating detailed explanations through multihop reasoning.

Original languageEnglish (US)
Title of host publicationEMNLP-IJCNLP 2019 - Graph-Based Methods for Natural Language Processing - Proceedings of the 13th Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages63-77
Number of pages15
ISBN (Electronic)9781950737864
StatePublished - Jan 1 2019
Event13th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2019, in conjunction with the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019 - Hong Kong, Hong Kong
Duration: Nov 4 2019Nov 4 2019

Publication series

NameEMNLP-IJCNLP 2019 - Graph-Based Methods for Natural Language Processing - Proceedings of the 13th Workshop

Conference

Conference13th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2019, in conjunction with the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019
CountryHong Kong
CityHong Kong
Period11/4/1911/4/19

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Information Systems

Fingerprint Dive into the research topics of 'TextGraphs 2019 shared task on multi-hop inference for explanation regeneration'. Together they form a unique fingerprint.

  • Cite this

    Jansen, P., & Ustalov, D. (2019). TextGraphs 2019 shared task on multi-hop inference for explanation regeneration. In EMNLP-IJCNLP 2019 - Graph-Based Methods for Natural Language Processing - Proceedings of the 13th Workshop (pp. 63-77). (EMNLP-IJCNLP 2019 - Graph-Based Methods for Natural Language Processing - Proceedings of the 13th Workshop). Association for Computational Linguistics (ACL).