Journal article
Automatically Identifying theQuality of Developer Chats for Post Hoc Use
ACM transactions on software engineering and methodology, v 30(4), 48
01 Jul 2021
Abstract
Software engineers are crowdsourcing answers to their everyday challenges on Q&A forums (e.g., Stack Over-flow) and more recently in public chat communities such as Slack, IRC, and Gitter. Many software-related chat conversations contain valuable expert knowledge that is useful for both mining to improve programming support tools and for readers who did not participate in the original chat conversations. However, most chat platforms and communities do not contain built-in quality indicators (e.g., accepted answers, vote counts). Therefore, it is difficult to identify conversations that contain useful information for mining or reading, i.e., conversations of post hoc quality. In this article, we investigate automatically detecting developer conversations of post hoc quality from public chat channels. We first describe an analysis of 400 developer conversations that indicate potential characteristics of post hoc quality, followed by a machine learning-based approach for automatically identifying conversations of post hoc quality. Our evaluation of 2,000 annotated Slack conversations in four programming communities (python, clojure, elm, and racket) indicates that our approach can achieve precision of 0.82, recall of 0.90, F-measure of 0.86, and MCC of 0.57. To our knowledge, this is the first automated technique for detecting developer conversations of post hoc quality.
Metrics
Details
- Title
- Automatically Identifying theQuality of Developer Chats for Post Hoc Use
- Creators
- Preetha Chatterjee - University of DelawareKostadin Damevski - Virginia Commonwealth UniversityNicholas A. Kraft - UserVoice, 234 Fayetteville St,3rd Floor, Raleigh, NC 27601 USALori Pollock - University of Delaware
- Publication Details
- ACM transactions on software engineering and methodology, v 30(4), 48
- Publisher
- Assoc Computing Machinery
- Number of pages
- 28
- Grant note
- 1812968; 1813253 / National Science Foundation; National Science Foundation (NSF)
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Web of Science ID
- WOS:000683039100008
- Scopus ID
- 2-s2.0-85112088835
- Other Identifier
- 991021883914804721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Computer Science, Software Engineering