Covariate-aware transformer architectures for long-horizon demand forecasting in Retail analytics

Demand forecasting from a retail perspective affects various facets of the business, including workforce planning, inventory control, and supply chain management, and informs key business strategies. The long-range demand forecasting techniques evaluate various external influences and deal with time series data that often exhibit irregular patterns and complex seasonality, and are often hierarchical. In this work, we proposed a long-range demand forecasting framework that used covariates and was based on transformer architectures with the following key components: multihead self-attention, direct multi-horizon prediction, positional encodings, and categorical embeddings. This framework can model various temporal patterns and was context-aware within the retail system. Having implemented the framework, we benchmarked it against state-of-the-art demand forecasting models on several retail forecasting datasets, as well as other complex time-series, multi-dimensional forecasting datasets. The model empirically demonstrated a 3.1% improvement on the retail forecasting benchmark (achieving a weighted root mean squared scaled error (WRMSSE) of 0.498) and a 1.7% improvement on the benchmark for complex time series datasets (achieving an symmetric mean absolute percentage error (sMAPE) of 12.47) compared to the leading transformer models. Additionally, we provided empirical evidence of enhanced demand forecasting, improved stability and robustness, reduced training time, and improved model throughput. With all of these model features and enhancements combined, we believe this will make retail demand forecasting significantly more feasible and add considerable operational value, particularly for the planning and strategy aspects of retail management.