LTBoost: Boosted Hybrids of Ensemble Linear and Gradient Algorithms for the Long-term Time Series Forecasting

The progress of deep-learning-based forecasting architectures is evident through their expanding parameter configurations. However, the need for rapid online decision making in practical scenarios calls for an alternative strategy, highlighting the necessity for networks that are not only adaptive but also efficient in real-time operations. This shift is critical as we confront three principal challenges in deep-learning-based forecasting frameworks: (i) the inherent limitations of transformers, which, despite their attempts to preserve ordering information, the temporal information loss due to the permutation-invariant nature of self-attention mechanisms is inevitable, (ii) the inefficacy of linear models in capturing the dynamic interactions within swiftly evolving signals; and (iii) the incapacity of tree-based approaches to extrapolating beyond values present in the training set. In response to these challenges, we introduce LTBoost, an innovative boosted hybrid of linear and tree-based ensemble gradient algorithms tailored for long-term time series forecasting (LTSF) tasks, scalable to high data dimensions. LTBoost employs a dual strategy, beginning with a linear regression model to capture trends and extrapolate beyond known data, complemented by a robust nonlinear tree-based model that focuses on the residuals. This boosted hybrid approach not only addresses the challenges posed by existing models but also significantly improves forecast accuracy. The effectiveness of LTBoost is validated through empirical experiments conducted on nine well-established benchmark datasets, demonstrating superior performance and achieving state-of-the-art results in 32 out of 36 cases, measured by mean absolute error (MAE). Our findings also explore the impact of lag features and signal normalization techniques, demonstrating further improvements in predictive accuracy. This hybrid and highly effective approach highlights LTBoost's innovation and its resolution of specific forecasting challenges, setting the stage for its contribution to the field of time series forecasting, paving the way for its application in diverse real-world scenarios.