Simulation modeling and analysis
Download
1 / 33

Simulation Modeling and Analysis - PowerPoint PPT Presentation


  • 62 Views
  • Uploaded on

Simulation Modeling and Analysis. Session 12 Comparing Alternative System Designs. Outline. Comparing Two Designs Comparing Several Designs Statistical Models Metamodeling. Comparing two designs. Let the average measures of performance for designs 1 and 2 be  1 and  2 .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Simulation Modeling and Analysis' - karleigh-carpenter


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Simulation modeling and analysis

Simulation Modeling and Analysis

Session 12

Comparing Alternative System Designs


Outline
Outline

  • Comparing Two Designs

  • Comparing Several Designs

  • Statistical Models

  • Metamodeling


Comparing two designs
Comparing two designs

  • Let the average measures of performance for designs 1 and 2 be 1 and 2.

  • Goal of the comparison: Find point and interval estimates for 1 - 2


Example
Example

  • Auto inspection system design

  • Arrivals: E(6.316) min

  • Service:

    • Brake check N(6.5,0.5) min

    • Headlight check N(6,0.5) min

    • Steering check N(5.5,0.5) min

  • Two alternatives:

    • Same service person does all checks

    • A service person is devoted to each check


Comparing two designs contd
Comparing Two Designs -contd

  • Run length (ith design ) = Tei

  • Number of replications (ith design ) = Ri

  • Average response time for replication r (ith design = Yri

  • Averages and standard deviations over all replications, Y1* = S Yri / Ri and Y2* , are unbiased estimators of 1 and 2.


Possible outcomes
Possible outcomes

  • Confidence interval for 1 - 2 well to the left of zero. I.e. most likely 1 < 2.

  • Confidence interval for 1 - 2 well to the right of zero. I.e. most likely 1 > 2.

  • Confidence interval for 1 - 2 contains zero. I.e. most likely 1 ~ 2.

  • Confidence interval

    (Y1* - Y2*) ± t /2, s.e.(Y1* - Y2*)


Independent sampling with equal variances
Independent Sampling with Equal Variances

  • Different and independent random number streams are used to simulate the two designs.

    Var(Yi*) = var(Yri)/Ri = i2/Ri

    Var(Y1* - Y2*) = var(Y1*) + var(Y2*)

    = 12/R1 + 22/R2= VIND

  • Assume the run lengths can be adjusted to produce 12 ~22


Independent sampling with equal variances contd
Independent Sampling with Equal Variances -contd

  • Then Y1* - Y2* is a point estimate of 1 - 2

    Si2 =  (Yri - Yi*)2/(Ri - 1)

    Sp2 = [(R1-1) S12 + (R2-1) S22]/(R1+R2-2)

    s.e.(Y1*-Y2*) = Sp (1/R1 + 1/R2)1/2

     = R1 + R2 -2


Independent sampling with unequal variances
Independent Sampling with Unequal Variances

s.e.(Y1*-Y2*) = (S12/R1 + S22/R2)1/2

 = (S12/R1 + S22/R2)2/M

where

M = (S12/R1)2/(R1-1) + (S22/R2)2/(R2-1)

  • Here R1 and R2 must be > 6


Correlated sampling
Correlated Sampling

  • Correlated sampling induces positive correlation between Yr1 and Yr2 and reduces the variance in the point estimator of Y1*-Y2*

  • Same random number streams used for both systems for each replication r (R1 = R2 = R)

  • Estimates Yr1 and Yr2 are correlated but Yr1 and Ys2 (r n.e. s) are mutually independent.


Recall covariance
Recall: Covariance

var(Y1* - Y2*) = var(Y1* ) + var(Y2* ) -

2 cov(Y1* , Y2* ) =

= 12/R + 22/R - 2 121 2/R = VCORR

= VIND - 2 121 2/R

Recall: definition of covariance

cov(X1,X2) = E(X1 X2) - m1 m2 =

= corr(X1 X2) s1 s2 =

= r s1 s2


Correlated sampling contd
Correlated Sampling -contd

  • Let Dr =Yr1 - Yr2

    D* = (1/R)  Dr = Y1* - Y2*

    SD2 = (1/(R - 1))  (Dr - D*)2

  • Standard error for the 100(1- )% confidence interval

    s.e.(D*) = s.e.(Y1* - Y2* ) = SD/ R

    (Y1* - Y2*) ± t /2, SD/ R


Correlated sampling contd1
Correlated Sampling -contd

  • Random Number Synchronization Guides

    • Dedicate a r.n. stream for a specific purpose and use as many streams as needed. Assign independent seeds to each stream at the beginning of each run.

    • For cyclic task subsystems assign a r.n. stream.

    • If synchronization is not possible for a subsystem use an independent stream.


Example auto inspection
Example: Auto inspection

An = interarrival time for vehicles n,n+1

Sn(1) = brake inspection time for vehicle n in model 1

Sn(2) = headlight inspection time for vehicle n in model 1

Sn(3) = steering inspection time for vehicle n in model 1

  • Select R = 10, Total_time = 16 hrs


Example auto inspection1
Example: Auto inspection

  • Independent runs

    -18.1 < 1-2 < 7.3

  • Correlated runs

    -12.3 < 1-2 < 8.5

  • Synchronized runs

    -0.5 < 1-2 < 1.3


Confidence intervals with specified precision
Confidence Intervals with Specified Precision

  • Here the problem is to determine the number of replications R required to achieve a desired level of precision e in the confidence interval, based on results obtained using Ro replications

    R = (t /2,Ro-1 SD/e)2


Comparing several system designs
Comparing Several System Designs

  • Consider K alternative designs

  • Performance measure i

  • Procedures

    • Fixed sample size

    • Sequential sampling (multistage)


Comparing several system designs contd
Comparing Several System Designs -contd

  • Possible Goals

    • Estimation of each i

    • Comparing i to a control 1

    • All possible comparisons

    • Selection of the best i


Bonferroni method for multiple comparisons
Bonferroni Method for Multiple Comparisons

  • Consider C confidence intervals 1-i

  • Overall error probability E = j

  • Probability all statements are true (the parameter is contained inside all C.I.’s)

    P  1 - E

  • Probability one or more statements are false

    P E


Example auto inspection contd
Example: Auto inspection (contd)

  • Alternative designs for addition of one holding space

    • Parallel stations

    • No space between stations in series

    • One space between brake and headlight inspection

    • One space between headlight and steering inspection


Bonferroni method for selecting the best
Bonferroni Method for Selecting the Best

  • System with maximum expected performance is to be selected.

  • System with maximum performance and maximum distance to the second best is to be selected.

    i - max j ij 


Bonferroni method for selecting the best contd
Bonferroni Method for Selecting the Best -contd

1.- Specify  ,  and R0

2.- Make R0 replications for each of the K systems

3.- For each system i calculate Yi*

4.- For each pair of systems i and j calculate Sij2 and select the largest Smax2

5.-Calculate R = max{R0, t2 Smax2 / 2}

6.- Make R-R0 additional replications for each of the K systems

7.- Calculate overall means Yi** = (1/R)  Yri

8.-Select system with largest Yi** as the best


Statistical models to estimate the effect of design alternatives
Statistical Models to Estimate the Effect of Design Alternatives

  • Statistical Design of Experiments

    • Set of principles to evaluate and maximize the information gained from an experiment.

  • Factors (Qualitative and Quantitative), Levels and Treatments

  • Decision or Policy Variables.


Single factor randomized designs
Single Factor, Randomized Designs Alternatives

  • Single Factor Experiment

    • Single decision factor D ( k levels)

    • Response variable Y

    • Effect of level j of factor D, j

  • Completely Randomized Design

    • Different r.n. streams used for each replication at any level and for all levels.


Single factor randomized designs contd
Single Factor, Randomized Designs -contd Alternatives

  • Statistical model

    Yrj =  + j + rj

    where

    Yrj = observation r for level j

     = mean overall effect

    j = effect due to level j

    rj = random variation in observation r at level j

    Rj= number of observations for level j


Single factor randomized designs contd1
Single Factor, Randomized Designs -contd Alternatives

  • Fixed effects model

    • levels of factors fixed by analyst

    • rj normally distributed

    • Null hypothesis H0: j = 0 for all j=1,2,..,k

    • Statistical test: ANOVA (F-statistic)

  • Random effects model

    • levels chosen at random

    • j normally distributed


Anova test
ANOVA Test Alternatives

  • Levels-replications matrix

  • Compute level means (over replications) Y.i* and grand mean Y..*

  • Variation of the response w.r.t. Y..*

    Yrj - Y..* = (Y.j* - Y..*) + (Yrj - Y.j*)

  • Squaring and summing over all r and j

    SSTOT = SSTREAT + SSE


Anova test contd
ANOVA Test -contd Alternatives

  • Mean square MSE = SSE/(R-k) is unbiased estimator of var(Y). I.e. E(MSE) = 2

  • Mean square MSTREAT = SSTREAT/(k-1) is also unbiased estimator of var(Y).

  • Test statistic

    F = MSTREAT / MSE

  • If H0 is true F has an F distribution with k-1 and R-k d.o.f.

  • Find critical value of the statistic F1-

  • Reject H0 if F > F1-


Metamodeling
Metamodeling Alternatives

  • Independent (design) variables xi, i=1,2,..,k

  • Output response (random) variable Y

  • Metamodel

    • A simplified approximation to the actual relationship between the xi and Y

    • Regression analysis (least squares)

    • Normal equations


Linear regression
Linear Regression Alternatives

  • One independent variable x and one dependent variable Y

  • For a linear relationship

    E(Y:x) = 0 + 1 x

  • Simple Linear Regression Model

    Y = 0 + 1 x + 


Linear regression contd
Linear Regression -contd Alternatives

  • Observations (data points)

    (xi,Yi) i=1,2,..,n

  • Sum of squares of the deviations i2

    L =i2 =  [ Yi - 0’ - 1(xi - x*)]2

  • Minimizing w.r.t 0’ and 1 find

    0’* =  Yi /n

    1*=  Yi (xi - x*)/  (xi - x*)2

    0* = 0’* - 1*x*


Significance testing
Significance Testing Alternatives

  • Null Hypothesis H0: 1 = 0

  • Statistic (n-2 d.o.f)

    t0 = 1*/(MSE/Sxx)

    where

    MSE = (Yi - Ypi)/(n-2)

    Sxx = xi2 - ( xi )2/n

  • H0 is rejected if |t0| > t/2,n-2


Multiple regression
Multiple Regression Alternatives

  • Models

    Y = 0 + 1 x1 + 2 x2 + ... + m xm + 

    Y = 0 + 1 x+ 2 x2 + 

    Y = 0 + 1 x1 + 2 x2 + 3 x1 x2 + 


ad