- 749 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Time Series Models' - salena

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Stochastic processes

Stationarity

White noise

Random walk

Moving average processes

Autoregressive processes

More general processes

TopicsTime series are an example of a stochastic or random process

A stochastic process is 'a statistical phenomenen that evolves in timeaccording to probabilistic laws'

Mathematically, a stochastic process is an indexed collection of random variables

Stochastic processesWe are concerned only with processes indexed by time, either discrete time or continuous time processes such asStochastic processes

We base our inference usually on a single observation or realization of the process over some period of time, say [0, T] (a continuous interval of time) or at a sequence of time points {0, 1, 2, . . . T}Inference

To describe a stochastic process fully, we must specify the all finite dimensional distributions, i.e. the joint distribution of of the random variables for any finite set of timesSpecification of a process

A simpler approach is to only specify the moments—this is sufficient if all the joint distributions are normal

The mean and variance functions are given by

Specification of a processBecause the random variables comprising the process are not independent, we must also specify their covarianceAutocovariance

Inference is most easy, when a process is stationary—its distribution does not change over time

This is strict stationarity

A process is weakly stationary if its mean and autocovariance functions do not change over time

StationarityThe autocovariance depends only on the time difference or lag between the two time points involvedWeak stationarity

It is useful to standardize the autocovariance function (acvf)

Consider stationary case only

Use the autocorrelation function (acf)

AutocorrelationThis is a purely random process, a sequence of independent and identically distributed random variables

Has constant mean and variance

Also

White noiseStart with {Zt} being white noise or purely random, mean zero, s.d. Z

{Xt} is a moving average process of order q (written MA(q)) if for some constants 0, 1, . . . q we have

Usually 0 =1

Moving average processesThe mean and variance are given by

The process is weakly stationary because the mean is constant and the covariance does not depend on t

Moving average processesIf the Zt's are normal then so is the process, and it is then strictly stationary

The autocorrelation is

Moving average processesIn order to ensure there is a unique MA process for a given acf, we impose the condition of invertibility

This ensures that when the process is written in series form, the series converges

For the MA(1) process Xt = Zt + Zt - 1, the condition is ||< 1

Moving average processesFor general processes introduce the backward shift operator B

Then the MA(q) process is given by

Moving average processesThe general condition for invertibility is that all the roots of the equation lie outside the unit circle (have modulus less than one)Moving average processes

Assume {Zt} is purely random with mean zero and s.d. z

Then the autoregressive process of order p or AR(p) process is

Autoregressive processesThe first order autoregression is

Xt = Xt - 1 + Zt

Provided ||<1 it may be written as an infinite order MA process

Using the backshift operator we have

(1 – B)Xt = Zt

Autoregressive processesAutoregressive processes

- From the previous equation we have

Autoregressive processes

- Then E(Xt) = 0, and if ||<1

Autoregressive processes

- The AR(p) process can be written as

This is for

for some 1, 2, . . .

This gives Xt as an infinite MA process, so it has mean zero

Autoregressive processesConditions are needed to ensure that various series converge, and hence that the variance exists, and the autocovariance can be defined

Essentially these are requirements that the i become small quickly enough, for large i

Autoregressive processesThe i may not be able to be found however.

The alternative is to work with the i

The acf is expressible in terms of the roots i, i=1,2, ...p of the auxiliary equation

Autoregressive processesThen a necessary and sufficient condition for stationarity is that for every i, |i|<1

An equivalent way of expressing this is that the roots of the equation

must lie outside the unit circle

Autoregressive processesAn ARMA process can be written in pure MA or pure AR forms, the operators being possibly of infinite order

Usually the mixed form requires fewer parameters

ARMA processesGeneral autoregressive integrated moving average processes are called ARIMA processes

When differenced say d times, the process is an ARMA process

Call the differenced process Wt. Then Wt is an ARMA process and

ARIMA processesThe model for Xt is non-stationary because the AR operator on the left hand side has d roots on the unit circle

d is often 1

Random walk is ARIMA(0,1,0)

Can include seasonal terms—see later

ARIMA processesWe have assumed that the mean is zero in the ARIMA models

There are two alternatives

mean correct all the Wt terms in the model

incorporate a constant term in the model

Non-zero meanOutline of the approach

Sample autocorrelation & partial autocorrelation

Fitting ARIMA models

Diagnostic checking

Example

Further ideas

TopicsThe approach is an iterative one involving

model identification

model fitting

model checking

If the model checking reveals that there are problems, the process is repeated

Box-Jenkins approachModels to be fitted are from the ARIMA class of models (or SARIMA class if the data are seasonal)

The major tools in the identification process are the (sample) autocorrelation function and partial autocorrelation function

ModelsUse the sample autocovariance and sample variance to estimate the autocorrelation

The obvious estimator of the autocovariance is

AutocorrelationThe sample autocovariances are not unbiased estimates of the autocovariances—bias is of order 1/N

Sample autocovariances are correlated, so may display smooth ripples at long lags which are not in the actual autocovariances

AutocovariancesCan use a different divisor (N-k instead of N) to decrease bias—but may increase mean square error

Can use jacknifing to reduce bias (to order 1/N )—divide the sample in half and estimate using the whole and both halves

Autocovariances2

More difficult to obtain properties of sample autocorrelation

Generally still biased

When process is white noise

E(rk) –1/N

Var( rk) 1/N

Correlations are normal for N large

AutocorrelationGives a rough test of whether an autocorrelation is non-zero

If |rk|>2/(N) suspect the autocorrelation at that lag is non-zero

Note that when examining many autocorrelations the chance of falsly identifying a non-zero one increases

Consider physical interpretation

AutocorrelationBroadly speaking the partial autocorrelation is the correlation between Xt and Xt+k with the effect of the intervening variables removed

Sample partial autocorrelations are found from sample autocorrelations by solving a set of equations known as the Yule-Walker equations

Partial autocorrelationPlot the autocorrelations and partial autocorrelations for the series

Use these to try and identify an appropriate model

Consider stationary series first

Model identificationFor a MA(q) process the autocorrelation is zero at lags greater than q, partial autocorrelations tail off in exponential fashion

For an AR(p) process the partial autocorrelation is zero at lags greater than p, autocorrelations tail off in exponential fashion

Stationary seriesFor mixed ARMA processes, both the acf and pacf will have large values up to q and p respectively, then tail off in an exponential fashion

See graphs in M&W, pp. 136–137

Try fitting a model and examine the residuals is the approach used

Stationary seriesThe existence of non-stationarity is indicated by an acf which is large at long lags

Induce stationarity by differencing

Differencing once is generally sufficient, twice may be needed

Overdifferencing introduces autocorrelation & should be avoided

Non-stationary seriesWe will always fit the model using Minitab

AR models may be fitted by least squares, or by solving the Yule-Walker equations

MA models require an iterative procedure

ARMA models are like MA models

EstimationMinitab uses an iterative least squares approach to fitting ARMA models

Standard errors can be calculated for the parameter estimates so confidence intervals and tests of significance can be carried out

MinitabBased on residuals

Residuals should be Normally distributed have zero mean, by uncorrelated, and should have minimum variance or dispersion

Diagnostic checkingPlot residuals against time

Draw histogram

Obtain normal scores plot

Plot acf and pacf of residuals

Plot residuals against fitted values

Note that residuals are not uncorrelated, but are approximately so at long lags

ProceduresBox and Peirce proposed a statistic which tests the magnitudes of the residual autocorrelations as a group

Their test was to compare Q below with the Chi-Square with K – p – q d.f. when fitting an ARMA(p, q) model

Portmanteau testBox & Ljung discovered that the test was not good unless n was very large

Instead use modified Box-Pierce or Ljung-Box-Pierce statistic—reject model if Q* is too large

Portmanteau testSuppose we think an AR(2) model is appropriate. We fit an AR(3) model.

The estimate of the additional parameter should not be significantly different from zero

The other parameters should not change much

This is an example of overfitting

OverfittingChatfield(1979), JRSS A among others has suggested the use of the inverse autocorrelation to assist with identification of a suitable model

Abraham & Ledolter (1984) Biometrika show that although this cuts off after lag p for the AR(p) model it is less effective than the partial autocorrelation for detecting the AR order

Other identification toolsThe Akaike Information Criterion is a function of the maximum likelihood plus twice the number of parameters

The number of parameters in the formula penalizes models with too many parameters

AICOnce principal generally accepted is that models should be parsimonious—having as few parameters as possible

Note that any ARMA model can be represented as a pure AR or pure MA model, but the number of parameters may be infinite

ParsimonyAR models are easier to fit so there is a temptation to fit a less parsimonious AR model when a mixed ARMA model is appropriate

Ledolter & Abraham (1981) Technometrics show that fitting unnecessary extra parameters, or an AR model when a MA model is appropriate, results in loss of forecast accuracy

ParsimonyMost exponential smoothing techniques are equivalent to fitting an ARIMA model of some sort

Winters' multiplicative seasonal smoothing has no ARIMA equivalent

Winters' additive seasonal smoothing has a very non-parsimonious ARIMA equivalent

Exponential smoothingFor example simple exponential smoothing is the optimal method of fitting the ARIMA (0, 1, 1) process

Optimality is obtained by taking the smoothing parameter to be 1 – when the model is

Exponential smoothing
Download Presentation

Connecting to Server..