From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442): Manifesto from Dagstuhl Perspectives Workshop 17442
Ferro N., Fuhr N., Grefenstette G., Konstan J. A., Castells P., Daly E. M., Declerck T., Ekstrand M. D., Geyer W., Gonzalo J., …
Published, Version of Record (VoR)CC BY V3.0, Open
Abstract
electrical engineering, electronic engineering, information engineering engineering and technology information & library sciences information systems other social sciences Social Sciences
We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of predic- tion models describing the relationship between assumptions, features and resulting performance
From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)