Logo image
Back
Sliding into My DMs: Detecting Uncomfortable or Unsafe Sexual Risk Experiences within Instagram Direct Messages Grounded in the Perspective of Youth
Journal article   Peer reviewed

Sliding into My DMs: Detecting Uncomfortable or Unsafe Sexual Risk Experiences within Instagram Direct Messages Grounded in the Perspective of Youth

Afsaneh Razi, Ashwaq Alsoubai, Seunghyun Kim, Shiza Ali, Gianluca Stringhini, Munmun De Choudhury and Pamela J. Wisniewski
Proceedings of the ACM on human-computer interaction, v 7(CSCW1), pp 1-29
16 Apr 2023
DOI: https://doi.org/10.1145/3579522
url
https://doi.org/10.1145/3579522View
Published, Version of Record (VoR) Restricted

Abstract

Artificial intelligence Classification and regression trees Collaborative and social computing Collaborative and social computing systems and tools Computing methodologies Empirical studies in collaborative and social computing Empirical studies in HCI Human and societal aspects of security and privacy Human computer interaction (HCI) Human-centered computing Machine learning Machine learning algorithms Machine learning approaches Natural language processing Neural networks Security and privacy Social aspects of security and privacy Social networking sites
We collected Instagram data from 150 adolescents (ages 13-21) that included 15,547 private message conversations of which 326 conversations were flagged as sexually risky by participants. Based on this data, we leveraged a human-centered machine learning approach to create sexual risk detection classifiers for youth social media conversations. Our Convolutional Neural Network (CNN) and Random Forest models outperformed in identifying sexual risks at the conversation-level (AUC=0.88), and CNN outperformed at the message-level (AUC=0.85). We also trained classifiers to detect the severity risk level (i.e., safe, low, medium-high) of a given message with CNN outperforming other models (AUC=0.88). A feature analysis yielded deeper insights into patterns found within sexually safe versus unsafe conversations. We found that contextual features (e.g., age, gender, and relationship type) and Linguistic Inquiry and Word Count (LIWC) contributed the most for accurately detecting sexual conversations that made youth feel uncomfortable or unsafe. Our analysis provides insights into the important factors and contextual features that enhance automated detection of sexual risks within youths' private conversations. As such, we make valuable contributions to the computational risk detection and adolescent online safety literature through our human-centered approach of collecting and ground truth coding private social media conversations of youth for the purpose of risk classification.

Metrics

13 Record Views
40 citations in Scopus

Details

Logo image