Conference proceeding
Beyond Citations: Corpus-based Methods for Detecting the Impact of Research Outcomes on Society
PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020), pp.6777-6785
01 Jan 2020
Abstract
This paper proposes, implements and evaluates a novel, corpus-based approach for identifying categories indicative of the impact of research via a deductive (top-down, from theory to data) and an inductive (bottom-up, from data to theory) approach. The resulting categorization schemes differ in substance. Research outcomes are typically assessed by using bibliometric methods, such as citation counts and patterns, or alternative metrics, such as references to research in the media. Shortcomings with these methods are their inability to identify impact of research beyond academia (bibliometrics) and considering text-based impact indicators beyond those that capture attention (altmetrics). We address these limitations by leveraging a mixed-methods approach for eliciting impact categories from experts, project personnel (deductive) and texts (inductive). Using these categories, we label a corpus of project reports per category schema, and apply supervised machine learning to infer these categories from project reports. The classification results show that we can predict deductively and inductively derived impact categories with 76.39% and 78.81% accuracy (F1-score), respectively. Our approach can complement solutions from bibliometrics and scientometrics for assessing the impact of research and studying the scope and types of advancements transferred from academia to society.
Metrics
2 Record Views
Details
- Title
- Beyond Citations: Corpus-based Methods for Detecting the Impact of Research Outcomes on Society
- Creators
- Rezvaneh Rezapour - University of Illinois at Urbana-ChampaignJutta Bopp - Leibniz Inst Deutsch Sprache, Mannheim, GermanyNorman Fiedler - Leibniz Inst Deutsch Sprache, Mannheim, GermanyDiana Steffen - Leibniz Inst Deutsch Sprache, Mannheim, GermanyAndreas Witt - Leibniz Inst Deutsch Sprache, Mannheim, GermanyJana Diesner - University of Illinois Urbana-Champaign
- Contributors
- N Calzolari (Editor)F Bechet (Editor)P Blache (Editor)K Choukri (Editor)C Cieri (Editor)T Declerck (Editor)S Goggi (Editor)H Isahara (Editor)B Maegaard (Editor)J Mariani (Editor)H Mazo (Editor)A Moreno (Editor)J Odijk (Editor)S Piperidis (Editor)
- Publication Details
- PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020), pp.6777-6785
- Conference
- 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020), 12th
- Publisher
- European Language Resources Assoc-Elra
- Number of pages
- 9
- Grant note
- 01IO1634 / German Federal Ministry of Education and Research (BMBF); Federal Ministry of Education & Research (BMBF)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Information Science (Informatics)
- Identifiers
- 991021861848704721
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- International collaboration
- Web of Science research areas
- Computer Science, Interdisciplinary Applications
- Language & Linguistics
- Linguistics