TY - GEN
T1 - TextGraphs 2019 shared task on multi-hop inference for explanation regeneration
AU - Jansen, Peter
AU - Ustalov, Dmitry
PY - 2019/1/1
Y1 - 2019/1/1
N2 - While automated question answering systems are increasingly able to retrieve answers to natural language questions, their ability to generate detailed human-readable explanations for their answers is still quite limited. The Shared Task on Multi-Hop Inference for Explanation Regeneration tasks participants with regenerating detailed gold explanations for standardized elementary science exam questions by selecting facts from a knowledge base of semistructured tables. Each explanation contains between 1 and 16 interconnected facts that form an "explanation graph" spanning core scientific knowledge and detailed world knowledge. It is expected that successfully combining these facts to generate detailed explanations will require advancing methods in multihop inference and information combination, and will make use of the supervised training data provided by the WorldTree explanation corpus. The top-performing system achieved a mean average precision (MAP) of 0.56, substantially advancing the state-of-the-art over a baseline information retrieval model. Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low, highlighting the difficulty of generating detailed explanations through multihop reasoning.
AB - While automated question answering systems are increasingly able to retrieve answers to natural language questions, their ability to generate detailed human-readable explanations for their answers is still quite limited. The Shared Task on Multi-Hop Inference for Explanation Regeneration tasks participants with regenerating detailed gold explanations for standardized elementary science exam questions by selecting facts from a knowledge base of semistructured tables. Each explanation contains between 1 and 16 interconnected facts that form an "explanation graph" spanning core scientific knowledge and detailed world knowledge. It is expected that successfully combining these facts to generate detailed explanations will require advancing methods in multihop inference and information combination, and will make use of the supervised training data provided by the WorldTree explanation corpus. The top-performing system achieved a mean average precision (MAP) of 0.56, substantially advancing the state-of-the-art over a baseline information retrieval model. Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low, highlighting the difficulty of generating detailed explanations through multihop reasoning.
UR - http://www.scopus.com/inward/record.url?scp=85085019634&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85085019634&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85085019634
T3 - EMNLP-IJCNLP 2019 - Graph-Based Methods for Natural Language Processing - Proceedings of the 13th Workshop
SP - 63
EP - 77
BT - EMNLP-IJCNLP 2019 - Graph-Based Methods for Natural Language Processing - Proceedings of the 13th Workshop
PB - Association for Computational Linguistics (ACL)
T2 - 13th Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs 2019, in conjunction with the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019
Y2 - 4 November 2019 through 4 November 2019
ER -