1 / 33

# Simulation Modeling and Analysis - PowerPoint PPT Presentation

Simulation Modeling and Analysis. Session 12 Comparing Alternative System Designs. Outline. Comparing Two Designs Comparing Several Designs Statistical Models Metamodeling. Comparing two designs. Let the average measures of performance for designs 1 and 2 be  1 and  2 .

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Simulation Modeling and Analysis' - karleigh-carpenter

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Simulation Modeling and Analysis

Session 12

Comparing Alternative System Designs

• Comparing Two Designs

• Comparing Several Designs

• Statistical Models

• Metamodeling

• Let the average measures of performance for designs 1 and 2 be 1 and 2.

• Goal of the comparison: Find point and interval estimates for 1 - 2

• Auto inspection system design

• Arrivals: E(6.316) min

• Service:

• Brake check N(6.5,0.5) min

• Steering check N(5.5,0.5) min

• Two alternatives:

• Same service person does all checks

• A service person is devoted to each check

• Run length (ith design ) = Tei

• Number of replications (ith design ) = Ri

• Average response time for replication r (ith design = Yri

• Averages and standard deviations over all replications, Y1* = S Yri / Ri and Y2* , are unbiased estimators of 1 and 2.

• Confidence interval for 1 - 2 well to the left of zero. I.e. most likely 1 < 2.

• Confidence interval for 1 - 2 well to the right of zero. I.e. most likely 1 > 2.

• Confidence interval for 1 - 2 contains zero. I.e. most likely 1 ~ 2.

• Confidence interval

(Y1* - Y2*) ± t /2, s.e.(Y1* - Y2*)

• Different and independent random number streams are used to simulate the two designs.

Var(Yi*) = var(Yri)/Ri = i2/Ri

Var(Y1* - Y2*) = var(Y1*) + var(Y2*)

= 12/R1 + 22/R2= VIND

• Assume the run lengths can be adjusted to produce 12 ~22

• Then Y1* - Y2* is a point estimate of 1 - 2

Si2 =  (Yri - Yi*)2/(Ri - 1)

Sp2 = [(R1-1) S12 + (R2-1) S22]/(R1+R2-2)

s.e.(Y1*-Y2*) = Sp (1/R1 + 1/R2)1/2

 = R1 + R2 -2

Independent Sampling with Unequal Variances

s.e.(Y1*-Y2*) = (S12/R1 + S22/R2)1/2

 = (S12/R1 + S22/R2)2/M

where

M = (S12/R1)2/(R1-1) + (S22/R2)2/(R2-1)

• Here R1 and R2 must be > 6

• Correlated sampling induces positive correlation between Yr1 and Yr2 and reduces the variance in the point estimator of Y1*-Y2*

• Same random number streams used for both systems for each replication r (R1 = R2 = R)

• Estimates Yr1 and Yr2 are correlated but Yr1 and Ys2 (r n.e. s) are mutually independent.

var(Y1* - Y2*) = var(Y1* ) + var(Y2* ) -

2 cov(Y1* , Y2* ) =

= 12/R + 22/R - 2 121 2/R = VCORR

= VIND - 2 121 2/R

Recall: definition of covariance

cov(X1,X2) = E(X1 X2) - m1 m2 =

= corr(X1 X2) s1 s2 =

= r s1 s2

• Let Dr =Yr1 - Yr2

D* = (1/R)  Dr = Y1* - Y2*

SD2 = (1/(R - 1))  (Dr - D*)2

• Standard error for the 100(1- )% confidence interval

s.e.(D*) = s.e.(Y1* - Y2* ) = SD/ R

(Y1* - Y2*) ± t /2, SD/ R

• Random Number Synchronization Guides

• Dedicate a r.n. stream for a specific purpose and use as many streams as needed. Assign independent seeds to each stream at the beginning of each run.

• For cyclic task subsystems assign a r.n. stream.

• If synchronization is not possible for a subsystem use an independent stream.

An = interarrival time for vehicles n,n+1

Sn(1) = brake inspection time for vehicle n in model 1

Sn(2) = headlight inspection time for vehicle n in model 1

Sn(3) = steering inspection time for vehicle n in model 1

• Select R = 10, Total_time = 16 hrs

• Independent runs

-18.1 < 1-2 < 7.3

• Correlated runs

-12.3 < 1-2 < 8.5

• Synchronized runs

-0.5 < 1-2 < 1.3

• Here the problem is to determine the number of replications R required to achieve a desired level of precision e in the confidence interval, based on results obtained using Ro replications

R = (t /2,Ro-1 SD/e)2

• Consider K alternative designs

• Performance measure i

• Procedures

• Fixed sample size

• Sequential sampling (multistage)

• Possible Goals

• Estimation of each i

• Comparing i to a control 1

• All possible comparisons

• Selection of the best i

• Consider C confidence intervals 1-i

• Overall error probability E = j

• Probability all statements are true (the parameter is contained inside all C.I.’s)

P  1 - E

• Probability one or more statements are false

P E

• Alternative designs for addition of one holding space

• Parallel stations

• No space between stations in series

• One space between brake and headlight inspection

• One space between headlight and steering inspection

• System with maximum expected performance is to be selected.

• System with maximum performance and maximum distance to the second best is to be selected.

i - max j ij 

1.- Specify  ,  and R0

2.- Make R0 replications for each of the K systems

3.- For each system i calculate Yi*

4.- For each pair of systems i and j calculate Sij2 and select the largest Smax2

5.-Calculate R = max{R0, t2 Smax2 / 2}

6.- Make R-R0 additional replications for each of the K systems

7.- Calculate overall means Yi** = (1/R)  Yri

8.-Select system with largest Yi** as the best

• Statistical Design of Experiments

• Set of principles to evaluate and maximize the information gained from an experiment.

• Factors (Qualitative and Quantitative), Levels and Treatments

• Decision or Policy Variables.

Single Factor, Randomized Designs Alternatives

• Single Factor Experiment

• Single decision factor D ( k levels)

• Response variable Y

• Effect of level j of factor D, j

• Completely Randomized Design

• Different r.n. streams used for each replication at any level and for all levels.

Single Factor, Randomized Designs -contd Alternatives

• Statistical model

Yrj =  + j + rj

where

Yrj = observation r for level j

 = mean overall effect

j = effect due to level j

rj = random variation in observation r at level j

Rj= number of observations for level j

Single Factor, Randomized Designs -contd Alternatives

• Fixed effects model

• levels of factors fixed by analyst

• rj normally distributed

• Null hypothesis H0: j = 0 for all j=1,2,..,k

• Statistical test: ANOVA (F-statistic)

• Random effects model

• levels chosen at random

• j normally distributed

ANOVA Test Alternatives

• Levels-replications matrix

• Compute level means (over replications) Y.i* and grand mean Y..*

• Variation of the response w.r.t. Y..*

Yrj - Y..* = (Y.j* - Y..*) + (Yrj - Y.j*)

• Squaring and summing over all r and j

SSTOT = SSTREAT + SSE

ANOVA Test -contd Alternatives

• Mean square MSE = SSE/(R-k) is unbiased estimator of var(Y). I.e. E(MSE) = 2

• Mean square MSTREAT = SSTREAT/(k-1) is also unbiased estimator of var(Y).

• Test statistic

F = MSTREAT / MSE

• If H0 is true F has an F distribution with k-1 and R-k d.o.f.

• Find critical value of the statistic F1-

• Reject H0 if F > F1-

Metamodeling Alternatives

• Independent (design) variables xi, i=1,2,..,k

• Output response (random) variable Y

• Metamodel

• A simplified approximation to the actual relationship between the xi and Y

• Regression analysis (least squares)

• Normal equations

Linear Regression Alternatives

• One independent variable x and one dependent variable Y

• For a linear relationship

E(Y:x) = 0 + 1 x

• Simple Linear Regression Model

Y = 0 + 1 x + 

Linear Regression -contd Alternatives

• Observations (data points)

(xi,Yi) i=1,2,..,n

• Sum of squares of the deviations i2

L =i2 =  [ Yi - 0’ - 1(xi - x*)]2

• Minimizing w.r.t 0’ and 1 find

0’* =  Yi /n

1*=  Yi (xi - x*)/  (xi - x*)2

0* = 0’* - 1*x*

Significance Testing Alternatives

• Null Hypothesis H0: 1 = 0

• Statistic (n-2 d.o.f)

t0 = 1*/(MSE/Sxx)

where

MSE = (Yi - Ypi)/(n-2)

Sxx = xi2 - ( xi )2/n

• H0 is rejected if |t0| > t/2,n-2

Multiple Regression Alternatives

• Models

Y = 0 + 1 x1 + 2 x2 + ... + m xm + 

Y = 0 + 1 x+ 2 x2 + 

Y = 0 + 1 x1 + 2 x2 + 3 x1 x2 + 