Understanding the Evaluation Metric
The accuracy competition introduced a new evaluation metric: Weighted Root Mean Squared Scaled Error (WRMSSE). You first start from the RMSSE of individual time series under scrutiny. The metric evaluates the deviation of the point forecasts around the mean of the realized values of the series being predicted:
where:
- n is the length of the training sample
- h is the forecasting horizon (in our case, it is h =28)
- Yt is the sales value at time t;
is the predicted value at time t
After estimating the RMSSE for all the 42,840 time series of the competition, the Weighted RMSSE will be computed as:
where wi is the weight of the ith series of the competition.
In the competition guidelines (https://mofc.unic.ac.cy/m5-competition/), in regard to RMSSE and WRMSSE, it is stated that:
- The denominator of RMSSE is computed only for the time periods for which the examined product(s) are actively sold...
