Logo image
Back
The Gray Area: Characterizing Moderator Disagreement on Reddit
Journal article   Open access

The Gray Area: Characterizing Moderator Disagreement on Reddit

Shayan Alipour, Shruti Phadke, Seyed Shahabeddin Mousavi, Amirhossein Afsharrad, Morteza Zihayat and Mattia Samory
Proceedings of the International AAAI Conference on Web and Social Media, v 20(1), pp 58-75
25 May 2026
DOI: https://doi.org/10.1609/icwsm.v20i1.42625
url
https://doi.org/10.1609/icwsm.v20i1.42625View
Published, Version of Record (VoR) Open

Abstract

Volunteer moderators play a crucial role in sustaining online dialogue, but they often disagree about what should or should not be allowed. In this paper, we study the complexity of content moderation with a focus on disagreements between moderators, which we term the “gray area” of moderation. Leveraging 5 years and 4.3 million moderation log entries from 24 subreddits of different topics and sizes, we characterize how gray area, or disputed cases, differ from undisputed cases. We show that one-in-seven moderation cases are disputed among moderators, often addressing transgressions where users' intent is not directly legible, such as in trolling and brigading, as well as tensions around community governance. This is concerning, as almost half of all gray area cases involved automated moderation decisions. Through extensive empirical analyses, we show that even state-of-the-art language models struggle to adjudicate gray area cases. Focusing on improving these models is unpromising. Through information-theoretic evaluations, we demonstrate that gray area cases are inherently harder to adjudicate than undisputed cases. We highlight the key role of expert human moderators in overseeing the moderation process and provide insights about the challenges of current moderation processes and tools.

Metrics

1 Record Views

Details

Logo image