Implementing time series cross validation to evaluate the forecasting model performance
Winita Sulandari(a*), Yudho Yudhanto(b), Sri Subanti(a), Etik Zukhronah(a), Muhammad Zidni Subarkah(a)

a) Study program of Statistics, Faculty of Mathematics and Natural Sciences, Universitas Sebelas Maret
*winita[at]mipa.uns.ac.id
b) Informatics Engineering, Vocational School, Universitas Sebelas Maret

Abstract

Theoretically, forecast error increases as the forecast horizon increases. This study aims to assess whether the statement is generally accepted or not. This study applies time series cross validation to evaluate forecasting results up to 7 steps ahead. As an illustration, we use Malaysia^s hourly electricity load data. Each hour is considered a series of each, so there are 24 daily series. Time series cross-validation with a 334 window was applied to twenty-four data series, and then each daily series was modeled with the Autoregressive Integrated Moving Average (ARIMA), Neural Network Autoregressive (NNAR), ExponenTial Smoothing (ETS), Singular Spectrum Analysis (SSA), and General Regression Neural Network (GRNN) models. In terms of mean absolute percentage error (MAPE) from 1 to 7 steps ahead, we then evaluate the performance of all models. The experimental results show that the MAPEs obtained from the GRNN model tend to increase along with the theory. However, MAPEs obtained from ETS increase by up to three steps ahead and decrease after that. Among the five models, ARIMA, NNAR, and SSA produce a reasonably stable MAPE value for 1 to 7 steps ahead. However, SSA has the most stable error value compared to ARIMA and NNAR.

Keywords: time series, cross validation, MAPE

Topic: Mathematics