Conference proceeding
Is "My Favorite New Movie" My Favorite Movie? Probing the Understanding of Recursive Noun Phrases
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 5286-5302
01 Jan 2022
Abstract
Recursive noun phrases (NPs) have interesting semantic properties. For example, my favorite new movie is not necessarily my favorite movie, whereas my new favorite movie is. This is common sense to humans, yet it is unknown whether language models have such knowledge. We introduce the Recursive Noun Phrase Challenge (RNPC), a dataset of three textual inference tasks involving textual entailment and event plausibility comparison, precisely targeting the understanding of recursive NPs. When evaluated on RNPC, state-of-the-art Transformer models only perform around chance. Still, we show that such knowledge is learnable with appropriate data. We further probe the models for relevant linguistic features that can be learned from our tasks, including modifier semantic category and modifier scope. Finally, models trained on RNPC achieve strong zero-shot performance on an extrinsic Harm Detection evaluation task, showing the usefulness of the understanding of recursive NPs in downstream applications.(1)
Metrics
5 Record Views
Details
- Title
- Is "My Favorite New Movie" My Favorite Movie? Probing the Understanding of Recursive Noun Phrases
- Creators
- Qing Lyu - University of PennsylvaniaZheng Hua - Peking UniversityDaoxin Li - University of PennsylvaniaLi Zhang - University of PennsylvaniaMarianna Apidianaki - University of PennsylvaniaChris Callison-Burch - University of Pennsylvania
- Publication Details
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 5286-5302
- Conference
- North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Seattle, WA, 10 Jul 2022–15 Jul 2022)
- Publisher
- Association for Computational Linguistics
- Number of pages
- 17
- Grant note
- 2019-19051600004 / IARPA BETTER Program FA8750-19-2-0201 / DARPA LwLL Program FA8750-19-2-1004 / DARPA KAIROS Program; United States Department of Defense
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science
- Web of Science ID
- WOS:000859869505030
- Other Identifier
- 991022123443104721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- International collaboration
- Web of Science research areas
- Computer Science, Artificial Intelligence
- Computer Science, Interdisciplinary Applications
- Linguistics