188 Views

Download Presentation
##### Time Series Analysis

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**School of Electrical Engineering**and Computer Science Time Series Analysis Topics in Machine Learning Fall 2011**Time Series Discussions**• Overview • Basic definitions • Time domain • Forecasting • Frequency domain • State space**Why Time Series Analysis?**• Sometimes the concept we want to learn is the relationship between points in time**What is a time series?**Time series: a sequence of measurements over time A sequence of random variables x1, x2, x3, …**Time Series Examples**Definition:A sequence of measurements over time • Finance • Social science • Epidemiology • Medicine • Meterology • Speech • Geophysics • Seismology • Robotics**Three Approaches**• Time domain approach • Analyze dependence of current value on past values • Frequency domain approach • Analyze periodic sinusoidal variation • State space models • Represent state as collection of variable values • Model transition between states**Sample Time Series Data**Johnson & Johnson quarterly earnings/share, 1960-1980**Sample Time Series Data**Yearly average global temperature deviations**Sample Time Series Data**Speech recording of “aaa…hhh”, 10k pps**Sample Time Series Data**NYSE daily weighted market returns**Not all time data will exhibit strong patterns…**LA annual rainfall**…and others will be apparent**Canadian Hare counts**Time Series Discussions**• Overview • Basic definitions • Time domain • Forecasting • Frequency domain • State space**Definitions**• Mean • Variance variance mean **Definitions**• Covariance • Correlation**Correlation**Y Y Y X X X r = -1 r = -.6 r = 0 Y Y Y X X X r = +1 r = 0 r = +.3**Redefined for Time**Mean function Ergodic? Autocovariance lag Autocorrelation**Autocorrelation Examples**lag Positive lag Negative**Stationarity – When there is no relationship**• {Xt} is stationary if • X(t) is independent of t • X(t+h,t) is independent of t for each h • In other words, properties of each section are the same • Special case: white noise**Time Series Discussions**• Overview • Basic definitions • Time domain • Forecasting • Frequency domain • State space**Linear Regression**• Fit a line to the data • Ordinary least squares • Minimize sum of squared distances between points and line • Try this out at http://hspm.sph.sc.edu/courses/J716/demos/LeastSquares/LeastSquaresDemo.html y = x + **R2: Evaluating Goodness of Fit**• Least squares minimizes the combined residual • Explained sum of squares is difference between line and mean • Total sum of squares is the total of these two y = x + **R2: Evaluating Goodness of Fit**• R2, the coefficient of determination • 0 R2 1 • Regression minimizes RSS and so maximizes R2 y = x + **Linear Regression**• Can report: • Direction of trend (>0, <0, 0) • Steepness of trend (slope) • Goodness of fit to trend (R2)**What if a linear trend does not fit my data well?**• Could be no relationship • Could be too much local variation • Want to look at longer-term trend • Smooth the data • Could have periodic or seasonality effects • Add seasonal components • Could be a nonlinear relationship**Moving Average**• Compute an average of the last m consecutive data points • 4-point moving average is • Smooths white noise • Can apply higher-order MA • Exponential smoothing • Kernel smoothing**Power Load Data**53 week 5 week**Piecewise Aggregate Approximation**• Segment the data into linear pieces Interesting paper**ARIMA: Putting the pieces together**• Autoregressive model of order p: AR(p) • Moving average model of order q: MA(q) • ARMA(p,q)**ARIMA: Putting the pieces together**• Autoregressive model of order p: AR(p) • Moving average model of order q: MA(q) • ARMA(p,q)**ARIMA: Putting the pieces together**• Autoregressive model of order p: AR(p) • Moving average model of order q: MA(q) • ARMA(p,q)**ARIMA: Putting the pieces together**• Autoregressive model of order p: AR(p) • Moving average model of order q: MA(q) • ARMA(p,q) • A time series is ARMA(p,q) if it is stationary and**ARIMA (AutoRegressive Integrated Moving Average)**• ARMA only applies to stationary process • Apply differencing to obtain stationarity • Replace its value by incremental change from last value • A process xt is ARIMA(p,d,q) if • AR(p) • MA(q) • Differenced d times • Also known as Box Jenkins**Time Series Discussions**• Overview • Basic definitions • Time domain • Forecasting • Frequency domain • State space**Express Data as Fourier Frequencies**• Time domain • Express present as function of the past • Frequency domain • Express present as function of oscillations, or sinusoids**Time Series Definitions**• Frequency, , measured at cycles per time point • J&J data • 1 cycle each year • 4 data points (time points) each cycle • 0.25 cycles per data point • Period of a time series, T = 1/ • J&J, T = 1/.25 = 4 • 4 data points per cycle • Note: Need at least 2**Fourier Series**• Time series is a mixture of oscillations • Can describe each by amplitude, frequency and phase • Can also describe as a sum of amplitudes at all time points • (or magnitudes at all frequencies) • If we allow for mixtures of periodic series then Take a look**How Compute Parameters?**• Regression • Discrete Fourier Transform • DFTs represent amplitude and phase of series components • Can use redundancies to speed it up (FFT)**Breaking down a DFT**• Amplitude • Phase**Example**2 2 1 1 GBP GBP 0 2 0 -1 1 -1 GBP 0 -1 2 1 GBP 0 2 -1 1 2 GBP 0 1 GBP -1 0 -1 1 frequency 2 frequencies 3 frequencies 5 frequencies 10 frequencies 20 frequencies