1 / 94

# Forecasting - PowerPoint PPT Presentation

Forecasting. JY Le Boudec. Contents. What is forecasting ? Linear Regression Avoiding Overfitting Differencing ARMA models Sparse ARMA models Case Studies. 1. What is forecasting ?. Assume you have been able to define the nature of the load

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Forecasting' - odina

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Forecasting

JY Le Boudec

1

• What is forecasting ?

• Linear Regression

• Avoiding Overfitting

• Differencing

• ARMA models

• Sparse ARMA models

• Case Studies

2

• Assume you have been able to define the nature of the load

• It remains to have an idea about its intensity

• It is impossible to forecast without error

• The good engineer should

• Forecast what can be forecast

• Give uncertainty intervals

• The rest is outside our control

3

• Simple, for simple cases

• Based on extrapolating the explanatory variables

5

• In practice we estimate  from y, …, yt

• When computing the forecast, we pretend  is known, and thus make an estimation error

• It is hoped that the estimation error is much less than the confidence interval for forecast

• In the case of linear regression, the theorem gives the global error exactly

• In general, we won’t have this luxury

9

• A case where estimation error versus prediction uncertainty can be quantified

• Prediction interval if model is known

• Prediction interval accounting for estimation (t = 100 observed points)

11

• The best model is not necessarily the one that fits best

12

• This is the overfitting problem

13

• Method 1: use of test data

• Method 2: information criterion

14

18

• We need to understand how to use discrete filters.

• Example: write the Matlab command for

24

• Q: compute X back from Y

26

• This is all very intuitive

34

• A prediction without prediction intervals is only a small part of the story

• The financial crisis might have been avoided if investors had been aware of prediction intervals

37

Linear Regression with 3 parameters + variance

Assuming differenced data is iid

40

• When the differenced data appears stationary but not iid

42

ARIMA Process

50

• Called the Box-Jenkins method

• Difference the data until stationary

• Examine ACF to get a feeling of order (p,q)

• Fit an ARMA model using maximum likelihood

51

Apply Scientific Method

1. make stationary and normal (how ?)

2. bound orders p,q

3. fit an ARMA model to Yt - 

i.e. Yt -» ARMA

4. compute residuals and verify white noise and normal

Fitting an ARMA modelPb is :

given orders p,q

given (x1, …xn) (transformed data)

compute the parameters of an ARMA (p,q) model that maximizes the likelihood

Q:What are the parameters ?

A: the mean , the polynomial coefficients k and k , the noise variance 2

52

• Maximizing the likelihood is a non-linear optimization problems

• Usually solved by iterative, heuristic algorithms, may converge to a local maximummay not converge

• Some simple, non MLE, heuristics exist for AR or MA models

• Ex: fit the AR model that has the same theoretical ACF as the sample ACF

• Common practice is to bootstrap the optimization procedure by starting with a “best guess”

• AR or MA fit, using heuristic above

53

Best Model Order prediction error

55

Check the Residuals prediction error

56

Example prediction error

57

58 prediction error

Forecasting with ARMA prediction error

• Assume Yt is fitted to an ARMA process

• The prediction problem is: given Y1=y1,…,Yt=yt find the conditional distribution of Yt+h

• We know it is normal, with a mean that depends on (y1,…,yt) and a variance that depends only on the fitted parameters of the ARMA process

• There are many ways to compute this; it is readily done by Matlab

59

60 prediction error

61 prediction error

Forecasting Formulae for ARIMA prediction error

• Y = original data

• X = differenced data, fitted to an ARMA model

• Obtain point prediction for X usingwhatwejustsaw

• Apply Proposition 6.4.1 to obtain point prediction for Y

• Apply formula for predictioninterval

• There are severalothermethods, but theymay have numericalproblems. Seecomments in the lecture notes afterprop 6.5.2

62

63 prediction error

Improve Confidence Interval If Residuals are not Gaussian (but appear to be iid)

• Assume residuals are not gaussian but are iid

• How can we get confidence intervals ?

• Bootstrap by sampling from residuals

64

65 (but appear to be iid)

With (but appear to be iid)bootstrapfromresiduals

• With gaussian assumption

66

6. Sparse ARMA Models (but appear to be iid)

• Problem: avoid many parameters when the degree of the A and C polynomials is high, as in the previous example

• Based on heuristics

• Multiplicative ARIMA, constrained ARIMA

• Holt Winters

67

68 (but appear to be iid)

Holt Winters Model 1: EWMA (but appear to be iid)

69

70 (but appear to be iid)

72 (but appear to be iid)

73 (but appear to be iid)

74 (but appear to be iid)

75 (but appear to be iid)

76 (but appear to be iid)

77 (but appear to be iid)

Sparse models give less accurate predictions but have much fewer parameters and are simple to fit.

Constrained ARIMA

(corrected or not)

78

7. Case Studies fewer parameters and are simple to fit.

79

80 fewer parameters and are simple to fit.

81 fewer parameters and are simple to fit.

82 fewer parameters and are simple to fit.

83 fewer parameters and are simple to fit.

84 fewer parameters and are simple to fit.

85 fewer parameters and are simple to fit.

86 fewer parameters and are simple to fit.

87 fewer parameters and are simple to fit.

h = 1 fewer parameters and are simple to fit.

88

89 fewer parameters and are simple to fit.

h = 2 fewer parameters and are simple to fit.

90

91 fewer parameters and are simple to fit.

log fewer parameters and are simple to fit.

h = 1

92

Conclusion fewer parameters and are simple to fit.

• Forecasting is useful when savings matter; for example

• Save money on server space rental

• Save energy

• Capturing determinism is perhaps most important and easiest

• Prediction intervals are useful to avoid gross mistakes

• Re-scaling the data may help

• … à vous de jouer.

93

Joyeuses fewer parameters and are simple to fit.

Pâques !

94