Journal article
Rethinking the evaluation of news algorithms: aligning epistemic standards, user priorities and evaluation metrics in recommender system design
Frontiers in communication, v 11
01 Jun 2026
Abstract
As media organizations increasingly deploy recommender systems, these technologies play a growing role in shaping how individuals encounter and engage with news and public information online. Designing and evaluating these systems effectively, particularly in the news domain, requires attention to the challenges users face in forming knowledge and justified beliefs within complex digital information environments. To address this need, we draw on the normative concept of epistemic welfare, which offers a systematic, veritistically grounded approach to assessing how recommender systems support users’ capacity to access, interpret, and use information. Using large-scale online surveys in Belgium and Switzerland (N = 3,076), this study investigates how users prioritize distinct epistemic standards in algorithmic recommendations of news vs. entertainment from public service vs. private media across two structurally similar contexts, with entertainment recommendations serving as a comparative reference category. Results show a consistent prioritization of reliability across all contexts, including entertainment, while efficiency and speed are systematically deprioritized. We integrate theoretical and empirical insights into the Epistemic Welfare Evaluation Framework for News Recommender Systems (EWEF-NR), an exploratory conceptual framework that connects normative epistemic standards and empirical user priorities with established evaluation metrics. Bridging social scientific and computational perspectives, this paper provides an interdisciplinary user-centered approach for designing and assessing news recommender systems that strengthen users’ epistemic agency.
Metrics
1 Record Views
Details
- Title
- Rethinking the evaluation of news algorithms: aligning epistemic standards, user priorities and evaluation metrics in recommender system design
- Creators
- Michelle Kulig (Corresponding Author) - University of FribourgWillem Buyens - University of AntwerpRaphaël Tuor - University of FribourgCélina Treuillier - University of FribourgManuel Puppis - University of FribourgHilde Van den Bulck - Drexel UniversitySteve Paulussen - University of AntwerpDenis Lalanne - University of Fribourg
- Publication Details
- Frontiers in communication, v 11
- Publisher
- Frontiers Media S.A
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Communication
- Other Identifier
- 991022192896304721