Journal article
Large-change forecast accuracy: Reanalysis of M3-Competition data using receiver operating characteristic analysis
International journal of forecasting, v 29(2)
01 Apr 2013
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
This paper applies receiver operating characteristic (ROC) analysis to micro-level, monthly time series from the M3-Competition. Forecasts from competing methods were used in binary decision rules to forecast exceptionally large declines in demand. Using the partial area under the ROC curve (PAUC) criterion as a forecast accuracy measure and paired-comparison testing via bootstrapping, we find that complex univariate methods (including Flores-Pearce 2, ForecastPRO, Automat ANN, Theta, and SmartFCS) perform best for this purpose. The Kendall tau test of dependency for PAUC and a judgmental index of forecast method complexity provide further confirming evidence. We also found that decision-rule combination forecasts using three top methods generally perform better than the component methods, although not statistically so. The top methods for forecasting large declines match the top methods for conventional forecast accuracy in the M3-Competition's micro monthly time series, and therefore, evidence from the M3-Competition suggests that practitioners should use complex univariate forecast methods for operations-level forecasting, for both ordinary and large-change forecasts. (C) 2012 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.
Metrics
Details
- Title
- Large-change forecast accuracy: Reanalysis of M3-Competition data using receiver operating characteristic analysis
- Creators
- Wilpen L. Gorr - Carnegie Mellon UniversityMatthew J. Schneider - Cornell University
- Publication Details
- International journal of forecasting, v 29(2)
- Publisher
- Elsevier
- Number of pages
- 8
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Decision Sciences (and Management Information Systems)
- Web of Science ID
- WOS:000316524300005
- Scopus ID
- 2-s2.0-84872701833
- Other Identifier
- 991021852205104721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Economics
- Management