steady state statistical analysis l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Steady-State Statistical Analysis PowerPoint Presentation
Download Presentation
Steady-State Statistical Analysis

Loading in 2 Seconds...

play fullscreen
1 / 17

Steady-State Statistical Analysis - PowerPoint PPT Presentation


  • 427 Views
  • Uploaded on

Steady-State Statistical Analysis. By Dr. Jason Merrick. What We’ll Do . Statistical analysis of steady-state simulations Warm up and run length Truncated replications Batching in a single run Automatic run-time confidence intervals via batch means.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Steady-State Statistical Analysis' - starbuck


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
what we ll do
What We’ll Do ...
  • Statistical analysis of steady-state simulations
    • Warm up and run length
    • Truncated replications
    • Batching in a single run
    • Automatic run-time confidence intervals via batch means

Simulation with Arena - Steady-state Output Analysis

statistical analysis of steady state simulations
Statistical Analysis of Steady-State Simulations
  • Recall: Difference between terminating, steady-state simulations
    • Which is appropriate depends on model, study
  • Now, assume steady-state is desired
    • Be sure this is so, since running and analysis is a lot harder than for terminating simulations
  • Naturally, simulation run lengths can be long
    • Opportunity for different internal computation order
    • Can change numerical results
    • Underscores need for statistical analysis of output

Simulation with Arena - Steady-state Output Analysis

warm up and run length
Warm Up and Run Length
  • Most models start empty and idle
    • Empty: No entities present at time 0
    • Idle: All resources idle at time 0
    • In a terminating simulation this is OK if realistic
    • In a steady-state simulation, though, this can bias the output for a while after startup
      • Bias can go either way
      • Usually downward (results are biased low) in queueing-type models that eventually get congested
      • Depending on model, parameters, and run length, the bias can be very severe

Simulation with Arena - Steady-state Output Analysis

warm up and run length cont d
Warm Up and Run Length (cont’d.)
  • Remedies for initialization bias
    • Better starting state, more typical of steady state
      • Throw some entities around the model
      • Can be inconvenient to do this in the model
      • How do you know how many to throw and where? (This is what you’re trying to estimate in the first place.)
    • Make the run so long that bias is overwhelmed
      • Might work if initial bias is weak or dissipates quickly
    • Let model warm up, still starting empty and idle
      • Simulate module: Warm-Up Period (time units!)
      • “Clears” all statistics at that point for summary report, any cross-replication data saved with Statistics module’s Outputs area (but not Time-Persistent or Tallies)

Simulation with Arena - Steady-state Output Analysis

warm up and run length cont d6
Warm Up and Run Length (cont’d.)
  • Warm-up and run length times?
    • Most practical idea: preliminary runs, plots
    • Simply “eyeball” them
    • Statistics module, Time-Persistent and Tallies areas, then Plot with Output Analyzer
  • Be careful about variability — make multiple replications, superimpose plots
  • Also, be careful to note “explosions”
  • Model 5.1:
    • Run for 1 day (=1440 minutes), 4 replications
    • Save within-run Shipped parts flow time values

Simulation with Arena - Steady-state Output Analysis

warm up and run length cont d7
Warm Up and Run Length (cont’d.)
  • No explosions
  • All seem to be settling into steady state
  • Run length seems adequate to reach steady state
  • Hard to judge warm-up ...

Simulation with Arena - Steady-state Output Analysis

warm up and run length cont d8
Warm Up and Run Length (cont’d.)
  • “Crop” plots to time 0 - 5,000
    • Plot dialog, “Display Time from … to …”
  • Conservative warm-up: maybe 2,000
  • If measures disagreed, use max warm-up

Simulation with Arena - Steady-state Output Analysis

truncated replications
Truncated Replications
  • If you can identify appropriate warm-up and run-length times, just make replications as for terminating simulations
    • Only difference: Specify Warm-Up Period in Simulate module
    • Proceed with confidence intervals, comparisons, all statistical analysis as in terminating case

Simulation with Arena - Steady-state Output Analysis

collecting truncated replications
Collecting Truncated Replications
  • Model 5.1:
    • Warm-Up Period = 6 hours (=360 minutes)
    • Run length 1800 minutes (1440 proper + 360 warm-up)
    • 10 replications
    • Collect flowtimes for shipped, salvaged and scrapped parts
    • Statistics module, Outputs area entries to save summary statistics (averages) across replications

Simulation with Arena - Steady-state Output Analysis

truncated replications cont d
Truncated Replications (cont’d.)
  • Output Analyzer, Classical Confidence Intervals
  • Separate invocations due to different units
  • Interpretation for steady-state expectations here
  • Want smaller?
    • More reps, same length
    • Longer reps, same number of them

Simulation with Arena - Steady-state Output Analysis

batching in a single run
Batching in a Single Run
  • If model warms up very slowly, truncated replications can be costly
    • Have to “pay” warm-up on each replication
  • Alternative: Just one R E A L L Y long run
    • Only have to “pay” warm-up once
    • Problem: Have only one “replication” and you need more than that to form a variance estimate (the basic quantity needed for statistical analysis)
      • Big no-no: Use the individual points within the run as “data” for variance estimate
      • Usually correlated (not indep.), variance estimate biased

Simulation with Arena - Steady-state Output Analysis

batching in a single run cont d
Batching in a Single Run (cont’d.)
  • Break each output record from the run into a few large batches
    • Tally (discrete-time) outputs: Observation-based
    • Time-Persistent (continuous-time): Time-based
  • Take averages over batches as “basic” statistics for estimation: Batch means
    • Tally outputs: Simple arithmetic averages
    • Time-Persistent: Continuous-time averages
  • Treat batch means as IID
    • Key: batch size for low correlation (details in text)
    • Still might want to truncate (once, time-based)

Simulation with Arena - Steady-state Output Analysis

batching in a single run cont d14
Batching in a Single Run (cont’d.)
  • Picture for WIP (time-persistent):
    • For observation-based Tallies, just count points
  • To batch and analyze (details in text):
    • Statistics module, Time-Persistent, Tally areas to save within-run records (could be big files)
    • Output Analyzer, Analyze/Batch/Truncate or
      • Warning if batches are too small for IID
    • Get means .flt file; Classical C.I. as before

Simulation with Arena - Steady-state Output Analysis

collecting batch means
Collecting Batch Means
  • Model 5.1:
    • Warm-Up Period 6 hours (=360 minutes)
    • Run length 14760 minutes
      • 10 * 1440 proper + 360 warm-up
      • Saves 9 warm-ups of 360 minutes from terminating version
    • Collect flowtimes for shipped, salvaged and scrapped parts
    • Statistics module, Outputs area entries to save tallies through simulation run
    • Output Analyzer, Correlogram for 500 lags
    • Output Analyzer, Analyze/Batch/Truncate Obsn’s
      • Truncate Time 360, Batch Observations try 150, 200, 250, …
      • Save batch means to .flt file

Simulation with Arena - Steady-state Output Analysis

automatic run time confidence intervals via batch means
Automatic Run-Time Confidence Intervals via Batch Means
  • Arena will automatically attempt to form 95% confidence intervals on steady-state output measures via batch means
    • “Half Width” column in summary output
    • Ignore if you’re doing a terminating simulation
    • Uses internal rules for batch sizes (details in text)
    • Won’t report anything if your run is not long enough
      • “(Insuf)” if you don’t have the minimum amount of data Arena requires even to form a c.i.
      • “(Correl)” if you don’t have enough data to form nearly-uncorrelated batch means, required to be safe

Simulation with Arena - Steady-state Output Analysis

recommendations other methods for steady state analysis
Recommendations, Other Methods for Steady-State Analysis
  • What to do?
    • Frankly, try to avoid steady-state simulations
      • Look at goal of the study
    • If you really do want steady-state
      • First try warm-up and truncated replications
      • Automatic run-time batch-means c.i.’s
      • Batch-means c.i.’s “by hand” if necessary
  • Other methods, goals
    • Large literature on steady-state analysis

Simulation with Arena - Steady-state Output Analysis