Simulation Modeling and Analysis

1 / 31

# Simulation Modeling and Analysis - PowerPoint PPT Presentation

Simulation Modeling and Analysis. Output Analysis. 1. Outline. Stochastic Nature of Output Taxonomy of Simulation Outputs Measures of Performance Point Estimation Interval Estimation Output Analysis in Terminating Simulations Output Analysis in Steady-state Simulations. 2. Introduction.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Simulation Modeling and Analysis' - MikeCarlo

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Simulation Modeling and Analysis

Output Analysis

1

Outline
• Stochastic Nature of Output
• Taxonomy of Simulation Outputs
• Measures of Performance
• Point Estimation
• Interval Estimation
• Output Analysis in Terminating Simulations
• Output Analysis in Steady-state Simulations

2

Introduction
• Output Analysis
• Analysis of data produced by simulation
• Goal
• To predict system performance
• To compare alternatives
• Why is it needed?
• To evaluate the precision of the simulation performance parameter as an estimator

3

Introduction -contd
• Each simulation run is a sample point
• Attempts to increase the sample size by increasing run length may fail because of autocorrelation
• Initial conditions affect the output

4

Stochastic Nature of Output Data
• Model Input Variables are Random Variables
• The Model Transforms Input into Output
• Output Data are Random Variables
• Replications of a model run can be obtained by repeating the run using different random number streams

5

Example: M/G/1 Queue
• Average arrival rate Poisson with  = 0.1 per minute
• Service times Normal with  = 9.5 minutes and  = 1.75 minutes
• Runs
• One 5000 minute run
• Five 1000 minute runs w/ 3 replications each

6

Taxonomy of Simulation Outputs
• Terminating (Transient) Simulations
• Runs until a terminating event takes place
• Uses well specified initial conditions
• Runs continually or over a very long time
• Results must be independent of initial data
• Termination?
• What determines the type of simulation?

7

Examples: Non-terminating Systems
• Many shifts of a widget manufacturing process.
• Expansion in workload of a computer service bureau.

8

• Point estimator of  (of ) based on the simulation discrete-time output (Y1, Y2,.., Yn)

* = (1/n) i n Yi

• Unbiased point estimator

E(* ) = 

• Bias

b = E(* ) - 

10

• Point estimator of  (of ) based on the simulation continuous-time output (Y(t), 0 < t < Te)

* = (1/ Te) 0 Te Y(t) dt

• Unbiased point estimator

E(* ) = 

• Bias

b = E(* ) - 

11

• Variance and variance estimator

2() = true variance of point estimator 

2*() = estimator of variance of point estimator 

• Bias (in variance estimation)

B = E(2*() )/ 2()

12

Measures of Performance: Interval Estimation - contd
• If B ~ 1 then t = ( - )/ 2*() has t/2,f distribution (d.o.f. = f). I.e.
• A 100(1 - )% confidence interval for is

 - t/2,f2*() <  <  + t/2,f2*()

• Cases
• Statistically independent observations
• Statistically dependent observations (time series).

13

Measures of Performance: Interval Estimation - contd
• Statistically independent observations
• Sample variance

S2 = i n (Yi -  )2/(n-1)

• Unbiased estimator of 2()

2*() = S2 /n

• Standard error of the point estimator 

*() = S /n

14

Measures of Performance: Interval Estimation - contd
• Statistically dependent observations
• Variance of 

2() = (1/n2) i n j n cov(Yi , Yj )

• Lag k autocovariance

k = cov(Yi , Yi+k )

• Lag k autocorrelation

k = k0

15

Measures of Performance: Interval Estimation - contd
• Statistically dependent observations (contd)
• Variance of 

2() = (0 /n) [ 1 + 2k=1 n-1 (1- k/n) k] = (0 /n) c

• Positively autocorrelated time series (k > 0)
• Negatively autocorrelated time series (k < 0)
• Bias (in variance estimation)

B = E(S2/n )/ 2() = (n/c - 1)/(n-1)

16

Measures of Performance: Interval Estimation - contd
• Statistically dependent observations (contd)
• Cases
• Independent data k = 0, c = 1, B = 1
• Positively correlated data k > 0, c > 1, B < 1, S2/n is biased low (underestimation)
• Negatively correlated data k < 0, c < 1, B > 1, S2/n is biased high (overestimation)

17

Output Analysis for Terminating Simulations
• Method of independent replications
• n = Sample size
• Number of replications r=1,2,…,R
• Yji i-th observation in replication j
• Yji, Yjk are autocorrelated
• Yri, Ysk are statistically independent
• Estimator of mean (r =1,2,…,R)

r(1/nr) i nrYri

18

Output Analysis for Terminating Simulations - contd
• Confidence Interval (R fixed; discrete data)
• Overall point estimate

* = (1/R) 1 Rr

• Variance estimate

* (*) = [1/(R-1)R] 1 R(r

• Standard error of the point estimator 

*() = * (*)

19

Output Analysis for Terminating Simulations - contd
• Estimator and Interval (R fixed; continuous data)
• Estimator of mean (r =1,2,…,R)

r(1/Te) 0 Te Yr(t) dt

Overall point estimate

* = (1/R) 1 Rr

• Variance estimate

* (*) = [1/(R-1)R] 1 R(r

20

Output Analysis in Terminating Simulations - contd
• Confidence Intervals with Specified Precision
• Half-length confidence interval (h.l.)

h.l. = t/2,f2*() = t/2,f S/ R < 

• Required number of replications

R* > ( z /2 So/  )2

21

Output Analysis for Steady State Simulations
• Let (Y1, Y2,.., Yn) be an autocorrelated time series
• Estimator of the long run measure of performance  (independent of I.C.s)

 = lim n => (1/n) i n Yi

• Sample size n (or Te) is design choice.

22

Output Analysis for Steady State Simulations -contd
• Considerations affecting the choice of n
• Estimator bias due to initial conditions
• Desired precision of point estimator
• Budget/computer constraints

23

Output Analysis for Steady State Simulations -contd
• Initialization bias and Initialization methods
• Intelligent initialization
• Using actual field data
• Using data from a simpler model
• Use of phases in simulation
• Initialization phase (0 < t < To; for i=1,2,…,d)
• Data collection phase (To < t < Te; for i=d+1,d+2,…,n)
• Rule of thumb (n-d) > 10 d

24

Output Analysis for Steady State Simulations -contd
• Example M/G/1 queue
• Batched data
• Batched means
• Averaging batch means within a replication (I.e. along the batches)
• Averaging batch means within a batch (I.e. along the replications).

25

• Cases

1.- Yrj is an individual observation from within a replication

2.- Yrj is a batch mean of discrete data from within a replication

3.- Yrj is a batch mean of continuous data over a given interval

26

Steady State Simulations: Replication Method -contd
• Sample average for replication r of all (nondeleted) observations

Y*r(n,d) = Y*r = [1/(n-d)] j=d+1n Yrj

• Replication averages are independent and identically distributed RV’s
• Overall point estimator

Y*(n,d) = Y* = [1/R] r=1R Yr(n,d)

27

Steady State Simulations: Replication Method -contd
• Sample Variance

S2 = [1/(R-1)] r=1R (Y*r - Y*)

• Standard error = S/ R
• 100(1-)% Confidence interval

Y* - t /2,R-1 S/ R <  < Y* + t /2,R-1 S/ R

28

• Greater precision can be achieved by
• Increasing the run length
• Increasing the number of replications

29

• Single, long replication with batches
• Batch means treated as if they were independent
• Batch means (continuous)

Y*j = (1/m) (j-1)mjm Y(t) dt

• Batch means (discrete)

Y*j = (1/m) i=(j-1)mjm Yi

30

Steady State Simulations: Batch Size Selection Guidelines
• Number of batches < 30
• Diagnose correlation with lag 1 autocorrelation obtained from a large number of batch means from a smaller batch size
• For total sample size to be selected sequentially allow batch size and number of batches grow with run length.

31