A robust nowcasting algorithm – An adaptation of Harrison’s harmonic smoothing

1. A robust nowcasting algorithm –An adaptation of Harrison’s harmonic smoothing Prof George Djolov University of Stellenbosch Business School & Statistics South Africa

2. Outline • Harrison’s procedure • Robust nowcasting algorithm • Results • Conclusions

3. Harrison’s procedure • Extract the trend by calculating a moving average with length equal to the length of seasonality in the series. The purpose of this smoothing is to eliminate the seasonality and some of the noise of the series, while its trend and cyclicality remain. • Obtain first round seasonal estimates by taking the ratio of each observed value to its corresponding moving average value, and then for each period take the average of all ratios corresponding to that period. The purpose of this, by virtue of the averaging, is to arrive at less noisy seasonal estimates of the series, for every period. • Smooth the first round seasonal estimates by Fourier transformation. The purpose of this is to take out more of the noise of the seasonal estimates as well as to facilitate economical projections of the series when faced with low data volume.

4. Harrison’s procedure (continued) • Based on an arbitrary cut-off detect outliers in the first round seasonal estimates, and for the affected period replace the corresponding observation’s value by the value obtained from the product between its trend surrogate and the smoothed seasonal estimate. Follow this up by repeating all preceding and current steps to clear out any additional outliers, terminating at the second or third repetition. The purpose of this is to take out from the series any unusual points whose retention would otherwise lead to erratic projections. • Do an analysis of variance on the seasonal ratios. The purpose of this is to test the adequacy of the seasonal estimates, as a precursor to the adequacy of the seasonal fit in the projection of the series. • Produce a short-term projection for a period by taking the product of the projected smoothed trend of a series for that period and its smoothed seasonal estimate.

5. Robust Nowcasting Algorithm (RNA) • By Dodge’s sample size rule select the number of observations from what is available, to get the size of the nominal series for running the RNA; • Inspect the selected nominal series, i.e. the one including trend and seasonality, with Tukey’s control chart for any irregular or out-of-bound movements in the trend; • Edit these movements by Winsorization to stabilise the nominal series trend within its normal or extended range. Series values within the control limits are replaced with themselves, i.e. they are notionally Winsorized; • To further reduce noise after Winsorizing, first smooth the Winsorized nominal series by Hanning, then smooth this output by a Tukey 3-point running median, and lastly pass this latter output through one more round of Hanning. In each smoothing round copy the opening and closing points from the previous round;

6. Robust Nowcasting Algorithm (continued) • Verify the noise reduction impact of smoothing from an overlay plot between the triple smoothed nominal series and its observed counterpart. Proceed to extrapolate the triple smoothed nominal series by a 3-point Hann projection rule to get the in-range and out-of-range nowcasts inclusive of trend and seasonality; • Derive the estimates of seasonal variation by taking the Persons link relatives of the triple smoothed nominal series. Clear out any noise from these estimates by firstly smoothing them with a Tukey 3-point running median, and secondly smoothing this output with a Hann filter; • Get the medians of the double smoothed link relatives, and then obtain their initial chain relatives, closing off their calculation loop after reaching the first period;

7. Robust Nowcasting Algorithm (continued) • Remove any residual trend effects in seasonality by subtracting from each initial chain relative its pro-rated periodic increment of trend as determined from the difference between the closing off and starting chain relatives for the first period; • Take out any noise in the trend-removed chain relatives by smoothing them with a Hann filter, followed by Hann filtering this output, and then again by Hann filtering this latter output. At each filtering round copy the opening and closing points of the trend-removed chain relatives from the prior round. Verify the impact of smoothing on noise reduction by an overlay plot between the trend-removed chain relatives before and after the smoothing; • Form the estimates of seasonal variation by taking the product between each triple filtered trend-removed chain relative and the ratio of their expected periodic sum to their observed total;

8. Robust Nowcasting Algorithm (continued) • Remove any residual trend effects in seasonality by subtracting from each initial chain relative its pro-rated periodic increment of trend as determined from the difference between the closing off and starting chain relatives for the first period; • Take out any noise in the trend-removed chain relatives by smoothing them with a Hann filter, followed by Hann filtering this output, and then again by Hann filtering this latter output. At each filtering round copy the opening and closing points of the trend-removed chain relatives from the prior round. Verify the impact of smoothing on noise reduction by an overlay plot between the trend-removed chain relatives before and after the smoothing; • Form the estimates of seasonal variation by taking the product between each triple filtered trend-removed chain relative and the ratio of their expected periodic sum to their observed total;

9. Robust Nowcasting Algorithm (continued) • Perform de-seasonalisation by applying the unit ratio product of the seasonal variation estimates to the triple smoothed nominal series; • Reduce the noise in the resultant de-seasonalised series by smoothing it with a Tukey 3-point running median, followed by re-smoothing this output with a Hann filter with copy-over of the opening and closing points from the preceding smooth; • As before, verify the noise reduction impact of smoothing from an overlay plot between the double smoothed de-seasonalised series and its observed seasonally adjusted counterpart. Proceed to extrapolate the double smoothed de-seasonalised series by a 3-point Hann projection rule to get the in-range and out-of-range nowcasts inclusive of trend;

10. Robust Nowcasting Algorithm (continued) • Numerically determine the fit of the nominal and seasonally adjusted series nowcasts by taking their relative mean absolute error, Pearson correlation, and Lin’s bias coefficient. Supplement this by visually checking for the fit by the overlay plots between the series as well as their agreement line plots. • Update the nominal and seasonally adjusted nowcasts by repeating the above steps with the arrival of new data every period, except for the estimates of seasonal variation which are updated once only at the beginning of the nowcasting cycle. This is because incoming information for the periods is still to run its course.

11. Results • EC’s Working Group on Labour Market Statistics (2012): “The monthly unemployment news release is the most downloaded of Eurostat's regular news releases.” • From here, the luck of the draw led to Ireland’s monthly unemployment levels as the trial series for the RNA. The data used is this published monthly by Eurostat. • The expected relative mean absolute error of Harrison’s procedure is typically 26.4%. No information on its Finley index or its RMSE. • Things are different at present. RNA’s Finley index (out-of-range directional accuracy) is 82%, RMAE is 4.7%, and the RMSE is 8.1% both for out-of-range predictions.

12. Results (continued) Figure 1. Nowcasted and observed nominal series agreement – Irish monthly unemployment levels (000’s)

13. Results (continued) Figure 2. Nowcasted and observed de-seasonalised series agreement – Irish monthly unemployment levels (000’s)

14. Results (continued) Table 1. Diagnostic statistics of nowcasts for nominal and de-seasonalised series – Irish monthly unemployment levels (000’s), 2016 nowcasting cycle

15. Results (continued) Figure 3. Nowcasted predictions of nominal series over time – Irish monthly unemployment levels (000’s)

16. Results (continued) Figure 4. Nowcasted predictions of de-seasonalised series over time – Irish monthly unemployment levels (000’s)

17. Conclusions • Statistically sophisticated or complex methods do not necessarily produce or provide more reliable nowcasts than simpler ones; • The length of the projection horizon shapes the reliability of the method, with shorter lengths in Refiletring producing reliable results; • Decisions as to which nowcasting method to employ are affected by the reliability measure used to make the decision, i.e. different reliability measures produce different rankings about the performance of nowcasting methods; • Combined-type methods are likely to have forecasting reliability that on average outperforms single or individual-type methods.