Construction and Evaluation of Response Surface Designs Incorporating Bias from Model Misspecificati...
Download
1 / 33

Construction and Evaluation of Response Surface Designs Incorporating Bias from Model Misspecification - PowerPoint PPT Presentation


  • 147 Views
  • Uploaded on

Construction and Evaluation of Response Surface Designs Incorporating Bias from Model Misspecification. Connie M. Borror , Arizona State University West Christine M. Anderson-Cook , Los Alamos National Laboratory Bradley Jones , JMP SAS Institute. Motivation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Construction and Evaluation of Response Surface Designs Incorporating Bias from Model Misspecification' - zihna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Construction and Evaluation of Response Surface Designs Incorporating Bias from Model Misspecification

Connie M. Borror, Arizona State University West

Christine M. Anderson-Cook, Los Alamos National Laboratory

Bradley Jones, JMP SAS Institute


Motivation l.jpg
Motivation Incorporating Bias from Model Misspecification

  • Response surface design evaluation (and creation) assuming a particular model

    • Single number efficiencies

    • Prediction variance performance

    • Mean-squared error

  • Model misspecification?

    • What effect does this have on prediction and optimization?


Motivation3 l.jpg
Motivation Incorporating Bias from Model Misspecification

  • Examine effect of model misspecification

    • Expected squared bias

    • Prediction variance

    • Expected mean squared error

    • Using fraction of design space (FDS) plots and box plots

  • Evaluate designs based on the contribution of ESB relative to PV.


Scenario l.jpg
Scenario Incorporating Bias from Model Misspecification

  • Cuboidal regions

  • True form of the model is of higher order than the model being fit.

  • Examine

    • Response surface models when the true form is cubic

    • Screening experiment when the true form is full second order.


Model specifications l.jpg
Model Specifications Incorporating Bias from Model Misspecification

  • The model to be fit is

    Y = X11 + ε

    • X1 = n × p design matrix for the assumed form of the model

  • The true form of the model is

    Y = X11+ X22 + ε

    • X2 = n × q design matrix pertaining to those parameters (2) not present in the model to be fit (assumed model).


Model specifications6 l.jpg
Model Specifications Incorporating Bias from Model Misspecification

  • 2in general, are not fully estimable

  • Assume 2~ N(0, )


Criteria l.jpg
Criteria Incorporating Bias from Model Misspecification

  • Mean-squared error

  • Expected squared bias (ESB):

  • Expected MSE sum of PV and ESB


Fraction of design space fds plots l.jpg
Fraction of Design Space (FDS) Plots Incorporating Bias from Model Misspecification

  • Zahran, Anderson-Cook, and Myers (2003) scaled prediction variance values are plotted versus the fraction of the design space that has SPV at or below the given value

  • Adapt this to plot ESB and EMSE as well as PV.

  • We use FDS plots and box plots to assess the designs


Cases l.jpg
Cases Incorporating Bias from Model Misspecification

  • I. Two-factor response surface design

    • Assume a second-order model:

    • True form of the model is cubic:


Case i designs l.jpg
Case I Designs Incorporating Bias from Model Misspecification

  • Central Composite Design (CCD)

  • Quadratic I-optimal (Q I-opt)

  • Quadratic D-optimal (Q D-opt)

  • Cubic I-optimal (C I-opt)

  • Cubic D-optimal (C D-opt)

  • Cubic Bayes I-optimal (C Bayes I-opt)

  • Cubic Bayes D-optimal (C Bayes D-opt)


Case i l.jpg
Case I Incorporating Bias from Model Misspecification

  • CCD


Case i designs12 l.jpg
Case I Designs Incorporating Bias from Model Misspecification

  • CCD (ESB and EMSE performance as bias increases)


Case i designs13 l.jpg
Case I Designs Incorporating Bias from Model Misspecification

  • PV for all designs


Case i designs14 l.jpg
Case I Designs Incorporating Bias from Model Misspecification

  • ESB for all designs


Case i designs15 l.jpg
Case I Designs Incorporating Bias from Model Misspecification

  • EMSE for all designs


Case i designs16 l.jpg
Case I Designs Incorporating Bias from Model Misspecification

  • FDS for EMSE for all designs


Case ii l.jpg
Case II Incorporating Bias from Model Misspecification

  • Four factor response surface design

    • Assume a second-order model:

    • True form of the model is cubic:

    • 20 additional terms as we move from the second-order model to cubic.


Case ii designs l.jpg
Case II Designs Incorporating Bias from Model Misspecification

  • Six possible designs, with n = 27 runs

    • Central Composite Design (CCD)

    • Box Behnken Design (BBD)

    • Quadratic I-optimal (Q I-Opt)

    • Quadratic D-optimal (Q D-Opt)

    • Cubic Bayes I-optimal (C Bayes I-Opt)

    • Cubic Bayes D-optimal (C Bayes D-Opt)

      Note: Cubic I- and D-Optimal not possible with available size of design


Case ii19 l.jpg
Case II Incorporating Bias from Model Misspecification

  • PV for all designs


Case ii20 l.jpg
Case II Incorporating Bias from Model Misspecification

  • EMSE for all designs


Case ii21 l.jpg
Case II Incorporating Bias from Model Misspecification

FDS plot of EMSE for Four Factors


Case iii l.jpg
Case III Incorporating Bias from Model Misspecification

  • Eight-factor Screening Design

    • Assume a first-order model:

    • True form of the model is full second-order:


Case iii designs l.jpg
Case III Designs Incorporating Bias from Model Misspecification

  • 28-4 fractional factorial design with 4 center runs

  • D-optimal (for first order)

  • Bayes I-optimal (for second order)

  • Bayes D-optimal (for second order)


Case iii designs24 l.jpg
Case III Designs Incorporating Bias from Model Misspecification

  • The difference in the number of terms from the assumed to the true form of the models increases from 8 to 44.

    • We would expect bias to quickly dominate EMSE.


Case iii25 l.jpg
Case III Incorporating Bias from Model Misspecification

  • PV for all designs


Case iii26 l.jpg
Case III Incorporating Bias from Model Misspecification

  • ESB for all designs


Case iii27 l.jpg
Case III Incorporating Bias from Model Misspecification

  • EMSE for all designs


Design notes l.jpg
Design Notes Incorporating Bias from Model Misspecification

  • For the two-factor case:

    • The I-optimal and CCD were equivalent.

    • They performed the best based on minimizing the maximum EMSE

    • They performed the best based on prediction variance


Design notes29 l.jpg
Design Notes Incorporating Bias from Model Misspecification

  • For the four factor case,

    • the BBD was best based on EMSE criteria (in particular, the 95th percentile, median, mean)

      • when size of the coefficients of missing terms are moderate to large

      • The I-optimal design was competitive for this case only if small amounts of bias were present.

    • As the number of missing cubic terms increases, the BBD was best for EMSE.


Design notes30 l.jpg
Design Notes Incorporating Bias from Model Misspecification

  • I-optimal designs were highly competitive over 95% of the design region; not with respect to the maximum PV, ESB, and EMSE.

  • Cubic Bayesian designs did not perform well.


Design notes31 l.jpg
Design Notes Incorporating Bias from Model Misspecification

  • In the screening design example:

    • The D-optimal designs best if the assumed model is correct, but break down quickly if quadratic terms are in the model

      • Much more pronounced than in the response surface design cases.

    • Quadratic Bayesian I-optimal design was best based on mean, median, and 95th percentile of EMSE

    • The 28-4 fractional factorial design was best with respect to the maximum EMSE.

    • The 28-4 design was best for both PV and ESB when the PV and ESB contribution to the model were balanced.


Conclusions l.jpg
Conclusions Incorporating Bias from Model Misspecification

  • Appropriate design can strongly depend on the assumption that we know the true form of the underlying model

  • If we select designs carefully it is often possible to select a model that predicts well in the design space, and provide some protection against missing model terms.

  • The ESB approach to assessing the effect of missing terms provides is advantageous:

    • do not have to specify coefficient values for the true underlying model,

    • Instead, the relative size of the missing terms can be calibrated relative to the variance of the observations.


Conclusions33 l.jpg
Conclusions Incorporating Bias from Model Misspecification

  • Size of the bias variance relative to observational error needed to balance contributions from PV and ESB is highly dependent on the number of missing terms from the assumed model.

  • As the number of missing terms increases, the ability of designs to cope with the bias decreases substantially

    • different designs are able to handle this increasing bias differently.


ad