The competition (https://www.kaggle.com/competitions/m5-forecasting-accuracy) ran from March to June 2020 and over 7,000 participants took part in it on Kaggle. The organizers arranged it into two separate tracks, one for point-wise prediction (accuracy track) and another one for estimating reliable values at different confidence intervals (uncertainty track). Our focus in this chapter will be to try to replicate one of the best submissions for the accuracy track and also pave the way for the uncertainty track (since it is based on the predictions of the accuracy one).

Walmart provided the data. It consisted of 42,840 daily sales time series of items hierarchically arranged into departments, categories, and stores spread in three U.S. states (the time series are somewhat correlated with each other). Along with the sales, Walmart also provided accompanying information (exogenous variables, usually not often provided in forecasting problems...

The accuracy competition introduced a new evaluation metric: Weighted Root Mean Squared Scaled Error (WRMSSE). You first start from the RMSSE of individual time series under scrutiny. The metric evaluates the deviation of the point forecasts around the mean of the realized values of the series being predicted:

where:

• n is the length of the training sample
• h is the forecasting horizon (in our case, it is h =28)
• Y_t is the sales value at time t; is the predicted value at time t

After estimating the RMSSE for all the 42,840 time series of the competition, the Weighted RMSSE will be computed as:

where w_i is the weight of the i_th series of the competition.

In the competition guidelines (https://mofc.unic.ac.cy/m5-competition/), in regard to RMSSE and WRMSSE, it is stated that:

• The denominator of RMSSE is computed only for the time periods for which the examined product(s) are actively sold...

There are many solutions available for the competition, mostly found on the competition Kaggle discussions pages. The top five methods of both challenges have also been gathered and published (except one because of proprietary rights) by the competition organizers themselves: https://github.com/Mcompetitions/M5-methods (by the way, reproducing the results of the winning submissions was a prerequisite for the collection of a competition prize).

Noticeably, all the Kagglers that placed in the higher ranks of the competitions have used, as their unique model type or in blended/stacked in ensembles, LightGBM because of its lesser memory usage and speed of computations, which gave it an advantage in the competition because of the large amount of times series to process and predict. But there are also other reasons for its success. Contrary to classical methods based on ARIMA, it doesn’t require relying on the analysis...

The plan for replicating Monsaraida’s solution is to create a notebook customizable by input parameters to produce the necessary processed data for training and test datasets and the LightGBM models for predictions. The models, given data in the past, will be trained to learn to predict values in a specific number of days in the future. The best results can be obtained by having each model learn to predict the values in a specific week range in the future. Since we have to predict up to 28 days ahead, we need a model predicting from day +1 to day +7 in the future, then another one able to predict from day +8 to day +14, another from day +15 to +21, and finally, another one capable of handling predictions from day +22 to day +28. We will need a Kaggle notebook for each of these time ranges, thus we need four notebooks. Each of these notebooks will be trained to predict the future time span for each of the 10 stores that...

You can see an example of how we assembled the predictions for both the public and private leaderboards here:

• Public leaderboard example: https://www.kaggle.com/lucamassaron/m5-predict-public-leaderboard
• Private leaderboard example: https://www.kaggle.com/code/lucamassaron/m5-predict-private-leaderboard

What changes between the public and private submissions is just the different last training day: it determines what days we are going to predict. The public leaderboard notebook has the last training day set to 1,913, and the private one has it set to 1,941. You can actually, just for validation purposes, create other versions of the public version notebook using these dates for creating past holdout validation sets: [1885, 1857, 1829, 1577]. Hence the notebook will produce predictions that you can test locally for confirming the predictive capability of the model.

In this second chapter, we took on quite a complex time series competition, hence the easiest top solution we tried is actually fairly complex, and it requires coding quite a lot of processing functions. After going through the chapter, you should have a better idea of how to process time series and have them predicted using gradient boosting. Favoring gradient-boosting solutions over traditional methods when you have enough data, as with this problem, should help you create strong solutions for complex problems with hierarchical correlations, intermittent series, and availability of covariates such as events, prices, or market conditions.

In the following chapters, you will tackle even more complex Kaggle competitions, dealing with images and text. You will be amazed at how much you can learn by recreating top-scoring solutions and understanding their inner workings.

Join our book’s Discord space

Join our...