(728a) Ensemble Models for Univariate Time Series Forecasting

Univariate time series (UTS) are stochastic processes and poorly modeled by conventional time-dependent regression due to their strong historical dependence and periodic behavior. Statistical models, like ARIMA (AutoRegressive Integrated Moving Average) methods, spectral modeling, and other non-linear techniques have long since dominated the UTS modeling literature; however, in 2010, Ahmed et al. demonstrated that machine learning models like multilayer perceptron neural networks and Gaussian Processes can compete with and outperform ARIMA models on business and economic forecasting problems [1]. This work first benchmarks the performance of many statistical and machine learning UTS models on datasets derived from scientific, natural, and business sources in order to establish a better understanding of each model’s relative accuracy and utility.

This work also leverages its benchmarking framework to identify prospective ensemble models and to guide modelers to an effective methodology for their problem class and topical domain. The use of ensembles has been shown to improve UTS forecasting accuracy already. Gheyas and Smith found success unifying multiple Generalized Regression Neural Networks (GRNN) with a Combiner GRNN [2]. Lai et al. implemented a deep learning framework coupled with a standard ARIMA model capture both long- and short-term trends in multivariate time series [3]. In 2017, Cerqueirla et al. implemented a dynamic ensemble capable of adapting to regime changes in the data [4]. This work extends upon the existing literature by applying best subset selection to the ensemble learning problem.

[1] Ahmed, N.K., Atiya, A.F., Gayar, N.E., El-Shishiny, H., An Empirical Comparison of Machine Learning Models for Time Series Forecasting, Econometric Reviews, 29:594–6215, 2010.

[2] Gheyas, I.A., Smith, L.S., A novel neural network ensemble architecture for time series forecasting, Neurocomputing, 74: 3855–3864, 2011.

[3] Lai, G., Chang, W.-C., Yang, Y., Liu, H., Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks, arXiv:1703.07015v2 [cs.LS], 2017.

[4] Cerqueira, V., Torgo, L., Oliveira, M., Pfahringer, B., Dynamic and Heterogeneous Ensembles for Time Series Forecasting, IEEE International Conference on Data Science and Advanced Analytics, DSAA, 2017.