Journal article
Relevance theory and distributions of judgments in document retrieval
Information processing & management, v 53(5), pp 1080-1102
01 Sep 2017
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
This article extends relevance theory (RT) from linguistic pragmatics into information retrieval. Using more than 50 retrieval experiments from the literature as examples, it applies RT to explain the frequency distributions of documents on relevance scales with three or more points. The scale points, which judges in experiments must consider in addition to queries and documents, are communications from researchers. In RT, the relevance of a communication varies directly with its cognitive effects and inversely with the effort of processing it. Researchers define and/or label the scale points to measure the cognitive effects of documents on judges. However, they apparently assume that all scale points as presented are equally easy for judges to process. Yet the notion that points cost variable effort explains fairly well the frequency distributions of judgments across them. By hypothesis, points that cost more effort are chosen by judges less frequently. Effort varies with the vagueness or strictness of scale-point labels and definitions. It is shown that vague scales tend to produce U- or V-shaped distributions, while strict scales tend to produce right-skewed distributions. These results reinforce the paper's more general argument that RT clarifies the concept of relevance in the dialogues of retrieval evaluation. (C) 2017 Elsevier Ltd. All rights reserved.
Metrics
Details
- Title
- Relevance theory and distributions of judgments in document retrieval
- Creators
- Howard D. White - Drexel University
- Publication Details
- Information processing & management, v 53(5), pp 1080-1102
- Publisher
- Elsevier
- Number of pages
- 23
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- [Retired Faculty]
- Web of Science ID
- WOS:000407402200005
- Scopus ID
- 2-s2.0-85018944792
- Other Identifier
- 991019168177704721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Computer Science, Information Systems
- Information Science & Library Science