terminating statistical analysis n.
Download
Skip this Video
Download Presentation
Terminating Statistical Analysis

Loading in 2 Seconds...

play fullscreen
1 / 14

Terminating Statistical Analysis - PowerPoint PPT Presentation


  • 230 Views
  • Uploaded on

Terminating Statistical Analysis. By Dr. Jason Merrick. Statistical Analysis of Output Data: Terminating Simulations. Random input leads to random output (RIRO) Run a simulation (once) — what does it mean? Was this run “typical” or not? Variability from run to run (of the same model)?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Terminating Statistical Analysis' - asis


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
statistical analysis of output data terminating simulations
Statistical Analysis of Output Data: Terminating Simulations
  • Random input leads to random output (RIRO)
  • Run a simulation (once) — what does it mean?
    • Was this run “typical” or not?
    • Variability from run to run (of the same model)?
  • Need statistical analysis of output data
  • Time frame of simulations
    • Terminating: Specific starting, stopping conditions
    • Steady-state: Long-run (technically forever)
    • Here: Terminating

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

point and interval estimation
Point and Interval Estimation
  • Suppose we are trying to estimate an output measure E[Y] =  based upon a simulated sample Y1,…,Yn
  • We come up with an estimate
    • For instance
  • How good is this estimate?
    • Unbiased
    • Low Variance (possibly minimum variance)
    • Consistent
    • Confidence Interval

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

t distribution
T-distribution
  • The t-statistic is given by
    • If the Y1,…,Ynare normally distributed and then the t-statistic is t-distributed
    • If the Y1,…,Ynare not normally distributed, but then the t-statistic is approximately t-distributed thanks to the Central Limit Theorem
      • requires a reasonably large sample size n
    • We require an estimate of the variance of denoted

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

t distribution confidence interval
T-distribution Confidence Interval
  • An approximate confidence interval for is then
    • The center of the confidence interval is
    • The half-width of the confidence interval is
    • is the 100(/2)% percentile of a t-distribution with f degrees of freedom.

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

t distribution confidence interval1
T-distribution Confidence Interval
  • Case 1: Y1,…,Ynare independent
    • This is the case when you are making n independent replications of the simulations
      • Terminating simulations
      • Try and force this with steady-state simulations
    • Compute your estimate and then compute the sample variance
    • s2 is an unbiased estimator of the population variance, so s2/n is an unbiased estimator of with f = n-1 degrees of freedom

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

t distribution confidence interval2
T-distribution Confidence Interval
  • Case 2: Y1,…,Ynare not independent
    • This is the case when you are using data generated within a single simulation run
      • sequences of observations in long-run steady-state simulations
    • s2/n is a biased estimator of
    • Y1,…,Ynis an auto-correlated sequence or a time-series
    • Suppose that our point estimator for is , a general result from mathematical statistics is

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

t distribution confidence interval3
T-distribution Confidence Interval
  • Case 2: Y1,…,Ynare not independent
    • For n observations there are n2 covariances to estimate
    • However, most simulations are covariance stationary, that is for all i, j and k
    • Recall that k is the lag, so for a given lag, the covariance remains the same throughout the sequence
    • If this is the case then there are n-1 lagged covariances to estimate, denoted k and

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

time series examples
Time-Series Examples

Positively correlated sequence with lag 1

Positively correlated sequence with lags 1 & 2

Positively correlated, covariance non-stationary sequence

Negatively correlated sequence with lag 1

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

t distribution confidence interval4
T-distribution Confidence Interval
  • Case 2: Y1,…,Ynare not independent
    • What is the effect of this bias term?
    • For primarily positively correlated sequences B < 1, so the half-width of the confidence interval will be too small
      • Overstating the precision => make conclusions you shouldn’t
    • For primarily negatively correlated sequences B > 1, so the half-width of the confidence interval will be too large
      • Underestimating the precision => don’t make conclusions you should

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

strategy for terminating simulations
Strategy for Terminating Simulations
  • For terminating case, make IID replications
    • Simulate module: Number of Replications field
    • Check both boxes for Initialization Between Reps.
    • Get multiple independent Summary Reports
    • Different random seeds for each replication
  • How many replications?
    • Trial and error (now)
    • Approximate no. for acceptable precision
    • Sequential sampling
  • Save summary statistics (e.g. average, variance) across replications
    • Statistics Module, Outputs Area, save to files

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

half width and number of replications
Half Width and Number of Replications
  • Prefer smaller confidence intervals — precision
  • Notation:
  • Confidence interval:
  • Half-width =

Want this to be “small,” say

< h where h is prespecified

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

half width and number of replications1
Half Width and Number of Replications
  • To improve the half-width, we can
    • Increase the length of each simulation run and so increase the mi
    • What does increasing the run length do?
    • Increase the number of replications

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis

half width and number of replications cont d
Half Width and Number of Replications (cont’d.)
  • Set half-width = h, solve for
  • Not really solved for n (t, s depend on n)
  • Approximation:
    • Replace t by z, corresponding normal critical value
    • Pretend that current s will hold for larger samples
    • Get
  • Easier but different approximation:

s = sample standard

deviation from “initial”

number n0 of replications

n grows quadratically

as h decreases.

h0 = half width from “initial”

number n0 of replications

Simulation with Arena — Intermediate Modeling and Terminating Statistical Analysis