Conference proceeding
Analyzing Toxicity in Open Source Software Communications Using Psycholinguistics and Moral Foundations Theory
2025 IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE), pp 1-8
27 Apr 2025
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
Studies have shown that toxic behavior can cause contributors to leave, and hinder newcomers' (especially from underrepresented communities) participation in Open Source Software (OSS) projects. Thus, detection of toxic language plays a crucial role in OSS collaboration and inclusivity. Off-the-shelf toxicity detectors are ineffective when applied to OSS communications, due to the distinct nature of toxicity observed in these channels (e.g., entitlement and arrogance are more frequently observed on GitHub than on Reddit or Twitter). In this paper, we investigate a machine learning-based approach for the automatic detection of toxic communications in OSS. We leverage psycholinguistic lexicons, and Moral Foundations Theory to analyze toxicity in two types of OSS communication channels; issue comments and code reviews. Our evaluation indicates that our approach can achieve a significant performance improvement (up to 7% increase in F1 score) over the existing domain-specific toxicity detector. We found that using moral values as features is more effective than linguistic cues, resulting in 67.50% F1-measure in identifying toxic instances in code review data and 64.83% in issue comments. While the detection accuracy is far from accurate, this improvement demonstrates the potential of integrating moral and psycholinguistic features in toxicity detection models. These findings highlight the importance of context-specific models that consider the unique communication styles within OSS, where interpersonal and value-driven language dynamics differ markedly from general social media platforms. Future work could focus on refining these models to further enhance detection accuracy, possibly by incorporating community-specific norms and conversational context to better capture the nuanced expressions of toxicity in OSS environments.
Metrics
4 Record Views
Details
- Title
- Analyzing Toxicity in Open Source Software Communications Using Psycholinguistics and Moral Foundations Theory
- Creators
- Ramtin Ehsani - Drexel UniversityRezvaneh Rezapour - Drexel UniversityPreetha Chatterjee - Drexel University
- Publication Details
- 2025 IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE), pp 1-8
- Publisher
- IEEE
- Number of pages
- 8
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Information Science (Informatics); Computer Science (Computing); College of Computing and Informatics
- Web of Science ID
- WOS:001554509700001
- Scopus ID
- 2-s2.0-105009456995
- Other Identifier
- 991022057814804721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Computer Science, Artificial Intelligence
- Computer Science, Software Engineering
- Computer Science, Theory & Methods