Prediction of a nonlinear time series with feedforward neural networks

1. Prediction of a nonlinear time series with feedforward neural networks Mats Nikus Process Control Laboratory

2. The time series

3. A closer look

4. Another look

5. Studying the time series • Some features seem to reapeat themselves over and over, but not totally ”deterministically” • Lets study the autocovariance function

6. The autocovariance function

7. Studying the time series • The autocovariance function tells the same: There are certainly some dynamics in the data • Lets now make a phaseplot of the data • In a phaseplot the signal is plotted against itself with some lag • With one lag we get

8. Phase plot

9. 3D phase plot

10. The phase plots tell • Use two lagged values • The first lagged value describes a parabola • Lets make a neural network for prediction of the timeseries based on the findings.

11. The neural network ^ y(k+1) Lets try with 3 hidden nodes 2 for the ”parabola” and one for the ”rest” y(k) y(k-1)

12. Prediction results

13. Residuals (on test data)

14. A more difficult case • If the time series is time variant (i.e. the dynamic behaviour changes over time) and the measurement data is noisy, the prediction task becomes more challenging.

15. Phase plot for a noisy timevariant case

16. Residuals with the model

17. Use a Kalman-filter to update the weights • We can improve the predictions by using a Kalman-filter • Assume that the process we want to predict is described by

18. Kalman-filter • Use the following recursive equations The gradient needed in Ck is fairly simple to calculate for a sigmoidal network

19. Residuals

20. Neural network parameters

21. Henon series • The timeseries is actually described by