Loading in 2 Seconds...

SIMULATION MODELING AND ANALYSIS WITH ARENA T. Altiok and B. Melamed Chapter 9 Output Analysis

Loading in 2 Seconds...

- 383 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'SIMULATION MODELING AND ANALYSIS WITH ARENA T. Altiok and B. Melamed Chapter 9 Output Analysis' - andrew

Download Now**An Image/Link below is provided (as is) to download presentation**

Download Now

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

SIMULATION MODELING AND ANALYSIS WITH ARENA

T. Altiok and B. Melamed

Chapter 9

Output Analysis

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

- Output Analysis is the modeling stage concerned with
- simulation replications
- computing statistics from replications
- presenting statistics in textual or graphical format
- Output Analysis activities consist of the following activities:
- replication design
- estimation of performance metrics
- system analysis and experimentation

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

- Simulation models can be classified into two main classes, based on their time horizon:
- terminatingmodels
- steady-statemodels

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

- A terminating simulation model has a natural termination time
- for its replications
- the time horizon is inherent in the system and finite
- the modeler is interested in short-term system dynamics
- statistics are computed within the system’s natural time horizon
- The number of replications is a critical parameter of the associated Output Analysis
- it is the only means of controlling the sample size of any given estimator

.

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Steady-State Simulation Models

- A steady state simulation model has no natural termination time
- for its replications
- the time horizon could be potentially infinite
- the modeler is interested in long-term dynamics and statistics
- While long-term statistics are of interest, initial system conditions tend to bias its long-term statistics
- it, therefore, makes sense to start statistics collection after an initial period of system warm-up, namely, after the biasing effect of the initial conditions decays to insignificance
- the transient-state regime is characterized by statistics that vary as function of time, while the steady-state regime prevails when statistics stabilize and do not vary over time

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Sample Steady-State Simulation Output

Workstation throughput and average number of jobs as functions of time

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Steady-State Simulation Issues

- Since steady-state models have no natural termination time, how does one select a replication length?
- the replication can be stopped when statistics at the end of several successive increments are sufficiently close (e.g., within some difference, to be determined by the analyst)
- Since a warm-up period is needed to eliminate statistical bias, how does one select the warm-up length?
- in a similar vein, the length of a warm-up period is determined by observing experimentally when the time variability of the statistics of interest largely disappears

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Statistics Collection From Replications

- Suppose we are interested in a parameter of the system
- e.g., mean flow time or blocking probability
- The simulation will then be programmed to produce a (variate)
- estimator, , which evaluates to someestimate
- For example, let denote the random variable of flow time through the system of job in replication , and let be a realization of
- if the estimator of the mean flow time is the sample mean of flow times in each replication, then the corresponding replication estimates are
- where is the number of flow time observations recorded in replication

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Batch Means Statistics Collection

- The Batch Means method is a practical way of collecting multiple estimates from a single replication
- the idea is to group observations into batches, which are iid or
- approximately so, and then collect one estimate from each batch
- Let the observed history be a discrete sample
- the total of observations is divided into batches of size each, such that (the batch size is selected to be large enough so as to ensure that the corresponding estimators are iid or approximately so)
- this results in the following partition into sub-samples (batches)
- for each batch , an estimate is formed from the
- observations of that batch only
- the replication thus yields a set of estimates

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Point Estimation for Discrete Samples

- Let a replication collect observations from a discrete sample, where
- is a sequence of variates,
- is the corresponding sample of observations
- The estimator for the mean value is the sample mean
- The sample mean above is classified as a point estimator, because it estimates a scalar
- In a similar vein, the corresponding point estimate is obtained when the sample of observations (realizations) are substituted into the formula for the sample mean above

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Point Estimation for Continuous Samples

- Let a replication collect observations from a continuous sample over time, where
- is a stochastic process over some time interval
- is the corresponding sample of observations
- The estimator for the mean value is the time average point estimate
- so called because the variates involved are indexed by time

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Point Estimation for Continuous Samples (Cont.)

- Examples of time-continuous variates in queueing context
- the number of jobs, , at time in a queueing system (buffer and server)
- the server utilization process, , is defined as the indicator
- variate
- where when the server is busy, and when the server is idle
- the utilization statistic is the time average
- which is the fraction of time in that the server is busy, and as such is an estimator of the probability that the server is busy

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Point Estimation in Arena

- The Statistic module allows the user to obtain estimates for
- Tally statistics
- Time Persistent statistics
- Both Tally and Time Persistent statistics permit user access
- to a number of related Arena variables, such as
- TAVG(X) = tally average of variable X
- DAVG(X) = time average of variable X

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Point Estimation in Arena

- Consider a workstation, subject to failure, where
- jobs arrive with exponential inter-arrival times of mean 1 hour
- jobs have a fixed processing time of 0.75 hours
- the workstation goes through up/down cycles as follows:
- while busy, it fails randomly with exponentially distributed time-to-failure of mean 20 hours
- on failure, repair times are uniformly distributed between 1 and 5 hours
- Based on analytical calculations, we expect
- the throughput to be 1 job per hour (same as the arrival rate)
- the down time probability to be 0.1125
- the average job delay (queue delay) to be 4.11 hours per job
- [see Altiok (1997), Chapter 3]

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Point Estimation in Arena (Cont.)

- The results of 5 replications from a simulation of a workstation
- subject to failures are displayed in the table below
- Note that the estimates vary from replication to replication
- the underlying estimator is a random variable!
- Any of the values can be used to estimate the true (unknown) parameter
- but how confident can the modeler be in their accuracy?

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Interval Estimation

- Confidence Interval (CI) estimation quantifies the confidence
- (probability) that an interval “covers” the true (but unknown) statistic
- the boundaries of the confidence interval are estimated using appropriate point estimates
- therefore, those boundaries are random variables, and the CI is a random interval which varies across experiments (replications)!
- the modeler predetermines the desired probability that the CI “covers” the true statistic (the larger the probability, the wider the interval)
- Confidence intervals can be computed for
- terminating simulations
- steady-state simulations

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Intervals for Terminating Simulations

- Assume that to estimate a parameter by simulation,
- independent fixed-length replications of the model were run
- the runs produced a sample , where is the point estimate of in replication
- The pooled point estimator for is the sample mean across replications
- this estimator is a random variable with mean and variance
- thus, increasing the number of replications, , would decrease the variance of , and consequently, increase our confidence in the corresponding point estimate value,

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Intervals for Terminating Simulations (Cont.)

- We wish to quantify the confidence in an estimate of the true parameter by computing (at least approximately) the probability of events of the form
- where
- the estimators and define a (random) confidence interval for the true parameter,
- is the probability that the confidence interval does not include the true parameter,
- is a small probability, (often 0.01 or 0.05), called the significance level

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Intervals for Terminating Simulations (Cont.)

- By the Central Limit Theorem,
- the random variable in the estimator is approximately normally distributed with mean and variance
- It follows that
- From properties of the normal distribution,
- the approximation improves as the sample size, , increases
- we have
- (or at least approximately so, if the bias is tolerable)

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Intervals for Terminating Simulations (Cont.)

- It is known that has the standard normal distribution with quantiles , where

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Intervals for Terminating Simulations (Cont.)

- A confidence interval for the mean parameter, , at significance level is given by
- In case the variance is unknown, proceed as follows:
- estimate the variance by
- use the fact that the Student t distribution satisfies
- the requisite confidence interval is

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Intervals for Steady-State Simulations

- Consider confidence interval estimation for some mean
- in Batch Means setting with a single replication of batches (the underlying history may be discrete or continuous)
- The corresponding set of estimators, , gives rise to
- sample mean
- sample variance
- The confidence interval for at significance level is

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Confidence Interval Estimation in Arena

- Standard Arena output provides95% Batch Means confidence intervals for each replication
- computed for both Tally and Time Persistent statistics in terms of half widths under the Half Width column heading
- if, however, the estimated batch means are significantly dependent or the underlying sample history is too short to yield a sufficient number of batches, then that column will display the message (Correlated)
- Arena also supports the computation of confidence intervals from multiple replications in the Outputs element of the SIMAN summary report

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Output Analysis via Standard Arena Output

- As a working example, consider a single-machine finishing operation workstation that processes two types of parts, denoted by G_1 and G_2, where
- parts of type G_1 arrive according to an exponential inter-arrival time distribution with mean 2 hours, and each part has a fixed processing time of 1 hour
- parts of type G_2 arrive according to an exponential inter-arrival time distribution with mean 4 hours, and each part has a fixed processing time of 1.4 hours
- all parts are processed in FIFO order
- all part types have equal service priorities
- We wish to simulate the finishing operation for 10,000 hours in order to understand the behavior of the number of parts in the workstation buffer and the buffer delay for each part type

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Output Analysis via Standard Arena Output (Cont.)

Arena model of a finishing operation with two types of parts

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Observation Collection

Record module with Tally statistics for part delay in the buffer

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Observation Collection (Cont.)

Dialog boxes for the Set module to tally delay times for each part type (bottom) and its members (top)

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Observation Collection (Cont.)

Dialog box for theStatistic module with a Tally statistic for each part delay

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Output summary report for Resources

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Output Summary (Cont.)

Output summary report for Queues

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

- From the summary report we observe the following partial utilizations in the finishing machine:
- is utilization due to type G_1 parts workload
- is utilization due to type G_2 parts workload
- is utilization due to overall workload
- We, therefore, expect the probability of the finishing machine being in the Busy state to be around 0.85
- The estimated probability in the summary report for Resources is actually 0.84 over a replication of length 10,000 hours.

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Output Analysis (Cont.)

- The table below shows the behavior of three performance measures as functions of replication length
- Observe how the estimates appear to converge to respective limiting values as the simulation length increases
- convergence is indicated by the fact that the values appear to stabilize and change very little for the higher range of replication lengths
- Since we know the true value of machine utilization, we can take advantage of this knowledge in deciding on the smallest replication length that gives rise to sufficiently accurate estimates
- high accuracy is indicated in the table by low variability in the estimates as function of replication length
- we naturally seek the smallest value, since we would like to reduce the computational effort as much as possible

.

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Output Analysis (Cont.)

Summary statistics based on 10 replications

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Output Analysis via the Arena Output Analyzer

- The Arena Output Analyzer tool provides the following capabilities:
- Data transfer
- data import from ASCII files
- data export to ASCII files
- Statistical analysis
- batching, correlogram, and basic statistics
- point estimation and confidence interval estimation for means and standard deviations
- statistical tests for comparing parameters of different samples
- Graphing
- data plots and charts
- statistics plots and charts

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Data Collection

Tallied buffer delay observations for parts of type G_1

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Graphical statistics for parts of type G_1

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Batching Data for Independent Observations

Batch/Truncate dialog box

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Batching Data for Independent Observations (Cont.)

Batched observations stored in file : G_1_Delay_Batched.flt

Initial Observations Truncated : 0

Number of Batches : 499

Number of Observations Per Batch : 1000

Number of Trailing Obs'ns Truncated : 486

Estimate of Covariance Between Batches : -0.002459

Batch/Truncate summary report for delay times of G_1 type parts

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Confidence Intervals for Means and Variances

Confidence intervals for mean delays of parts of type G_1 and G_2

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

- The Analyze menu in the Arena Output Analyzer tool provides the following options:
- Compare Means for comparing the means of two samples drawn from two populations
- Compare Variances for comparing the variances of two samples drawn from two populations
- These options perform statistical analysis for testing statistically for the equality of means and variances, respectively
- For example, to test the null hypothesis that the means are equal,
- a confidence interval is sets up for their difference
- since under the null hypothesis the difference is zero, one accepts equality if the confidence interval includes 0, and rejects it, otherwise

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Dialog boxes for comparing mean buffer delays of type G_1 and G_2 parts

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Example: Testing for Equality of Means

Test results for the equality of mean buffer delays of type G_1 and G_2 parts

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Point Estimation for Correlation

- The Analyze menu in the Arena Output Analyzer tool provides
- the Correlogram option to gauge the statistical dependence among observations within a sample
- performs point estimation of the correlation between two samples by estimating the coefficient of correlation between the respective random variables
- This activity is called correlation analysis

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

- The left-hand correlogram
- was computed from
- simulation observations
- of successive delays
- The right-hand correlogram
- was computed from
- the sample means of
- batches of 1000 delays
- The two correlograms
- are strikingly different: the successive delays
- are strongly positively
- correlated, while the sample means of the batches are very nearly uncorrelated

Results of correlation analysis for type G_1 parts

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Parametric Analysis via the Arena Process Analyzer

- The term Parametric Analysis refers to the following set of activities:
- running a model multiple times with a different set of input parameters for each run
- then comparing the resultant performance measures
- Its purpose is to carry out to Sensitivity Analysis
- understand the impact of parameter changes on system behavior
- often use this understanding to find the optimal configuration (parameter set) with respect to one or more performance measures, or combination thereof
- In Arena Process Analyzer parlance
- input parameters are called controls
- the resultant performance measures are called responses
- a collection of controls and responses for a given set of runs is referred to as a scenario
- a collection of scenarios is termed an Arena project

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

A Process Analyzer window with an initial grid

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

The Process Analyzer GUI (Cont.)

A Process Analyzer window with a populated grid

Altiok / Melamed Simulation Modeling and Analysis with Arena

Chapter 9

Download Presentation

Connecting to Server..