Conference proceeding
Reasoning about Goals, Steps, and Temporal Ordering with WikiHow
PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP), pp 4630-4639
01 Jan 2020
Abstract
We propose a suite of reasoning tasks on two types of relations between procedural events: GOAL-STEP relations ("learn poses" is a step in the larger goal of "doing yoga") and STEP-STEP TEMPORAL relations ("buy a yoga mat" typically precedes "learn poses"). We introduce a dataset targeting these two relations based on wikiHow, a website of instructional how-to articles. Our human-validated test set serves as a reliable benchmark for commonsense inference, with a gap of about 10% to 20% between the performance of state-of-the-art transformer models and human performance. Our automatically-generated training set allows models to effectively transfer to out-of-domain tasks requiring knowledge of procedural events, with greatly improved performances on SWAG, Snips, and Story Cloze Test in zero-and few-shot settings.
Metrics
Details
- Title
- Reasoning about Goals, Steps, and Temporal Ordering with WikiHow
- Creators
- Li Zhang - University of PennsylvaniaQing Lyu - University of PennsylvaniaChris Callison-Burch - University of Pennsylvania
- Publication Details
- PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP), pp 4630-4639
- Publisher
- Association for Computational Linguistics
- Number of pages
- 10
- Grant note
- 2019-19051600004 / IARPA BETTER Program FA8750-19-2-1004 / DARPA KAIROS Program; United States Department of Defense
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science
- Web of Science ID
- WOS:000855160704062
- Other Identifier
- 991022123355804721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Computer Science, Artificial Intelligence