dc.contributor.author |
Boratko, M |
en |
dc.contributor.author |
Padigela, H |
en |
dc.contributor.author |
Mikkilineni, D |
en |
dc.contributor.author |
Yuvraj, P |
en |
dc.contributor.author |
Das, R |
en |
dc.contributor.author |
McCallum, A |
en |
dc.contributor.author |
Chang, M |
en |
dc.contributor.author |
Fokoue-Nkoutche, A |
en |
dc.contributor.author |
Kapanipathi, P |
en |
dc.contributor.author |
Mattei, N |
en |
dc.contributor.author |
Musa, R |
en |
dc.contributor.author |
Talamadupula, K |
en |
dc.contributor.author |
Witbrock, Michael |
en |
dc.contributor.editor |
Choi, E |
en |
dc.contributor.editor |
Seo, M |
en |
dc.contributor.editor |
Chen, D |
en |
dc.contributor.editor |
Jia, R |
en |
dc.contributor.editor |
Berant, J |
en |
dc.coverage.spatial |
Melbourne, Australia |
en |
dc.date.accessioned |
2019-09-12T22:19:53Z |
en |
dc.date.issued |
2018-07-19 |
en |
dc.identifier.citation |
Editors: Choi E, Seo M, Chen D, Jia R, Berant J. Machine Reading for Question Answering Proceedings of the Workshop. The Association for Computational Linguistics. 60-70. 19 Jul 2018 |
en |
dc.identifier.isbn |
978-1-948087-39-1 |
en |
dc.identifier.uri |
http://hdl.handle.net/2292/47660 |
en |
dc.description.abstract |
The recent work of Clark et al. introduces the AI2 Reasoning Challenge (ARC) and the associated ARC dataset that partitions open domain, complex science questions into an Easy Set and a Challenge Set. That paper includes an analysis of 100 questions with respect to the types of knowledge and reasoning required to answer them; however, it does not include clear definitions of these types, nor does it offer information about the quality of the labels. We propose a comprehensive set of definitions of knowledge and reasoning types necessary for answering the questions in the ARC dataset. Using ten annotators and a sophisticated annotation interface, we analyze the distribution of labels across the Challenge Set and statistics related to them. Additionally, we demonstrate that although naive information retrieval methods return sentences that are irrelevant to answering the query, sufficient supporting text is often present in the (ARC) corpus. Evaluating with human-selected relevant sentences improves the performance of a neural machine comprehension model by 42 points. |
en |
dc.description.uri |
http://arxiv.org/abs/1806.00358v2 |
en |
dc.publisher |
The Association for Computational Linguistics |
en |
dc.relation.ispartof |
56th Annual Meeting of the Association for Computational Linguistics |
en |
dc.relation.ispartofseries |
Machine Reading for Question Answering Proceedings of the Workshop |
en |
dc.rights |
Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher. |
en |
dc.rights.uri |
https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm |
en |
dc.rights.uri |
https://creativecommons.org/licenses/by/4.0/ |
en |
dc.subject |
cs.AI |
en |
dc.subject |
cs.AI |
en |
dc.subject |
cs.CL |
en |
dc.subject |
cs.IR |
en |
dc.title |
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset |
en |
dc.type |
Conference Item |
en |
pubs.begin-page |
60 |
en |
dc.rights.holder |
Copyright: The Association for Computational Linguistics |
en |
pubs.author-url |
https://aclweb.org/anthology/W18-2607 |
en |
pubs.end-page |
70 |
en |
pubs.finish-date |
2019-07-20 |
en |
pubs.start-date |
2018-07-15 |
en |
dc.rights.accessrights |
http://purl.org/eprint/accessRights/OpenAccess |
en |
pubs.subtype |
Proceedings |
en |
pubs.elements-id |
774210 |
en |
pubs.org-id |
Science |
en |
pubs.org-id |
School of Computer Science |
en |
pubs.arxiv-id |
1806.00358 |
en |
pubs.record-created-at-source-date |
2019-09-24 |
en |