variable selection for tailoring treatment n.
Skip this Video
Download Presentation
Variable Selection for Tailoring Treatment

Loading in 2 Seconds...

play fullscreen
1 / 36

Variable Selection for Tailoring Treatment - PowerPoint PPT Presentation

  • Uploaded on

Variable Selection for Tailoring Treatment. L. Gunter, J. Zhu & S.A. Murphy ASA, Nov 11, 2008. Outline. Motivation Need for Variable Selection Characteristics of a Tailoring Variable A New Technique for Finding Tailoring Variables Comparisons Discussion. Motivating Example.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Variable Selection for Tailoring Treatment' - donovan-williams

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
variable selection for tailoring treatment

Variable Selection for Tailoring Treatment

L. Gunter, J. Zhu & S.A. Murphy

ASA, Nov 11, 2008

  • Motivation
  • Need for Variable Selection
  • Characteristics of a Tailoring Variable
  • A New Technique for Finding Tailoring Variables
  • Comparisons
  • Discussion
simple example
Simple Example

Nefazodone - CBASP Trial



Nefazodone + Cognitive

Behavioral Analysis

System of Psychotherapy


50+ baseline covariates, both categorical and continuous

simple example1
Simple Example

Nefazodone - CBASP Trial

Which variables in X are important for tailoring the treatment?

  • We want to select the treatment that “optimizes” R
  • The optimal choice of treatment may depend on X
  • The optimal treatment(s) is given by
  • The value of d is
need for variable selection
Need for Variable Selection
  • In clinical trials many pretreatment variables are collected to improve understanding and inform future treatment
  • Yet in clinical practice, only the most informative variables for tailoring treatment can be collected.
  • A combination of theory, clinical experience and statistical variable selection methods can be used to determine which variables are important.
current statistical variable selection methods
Current Statistical Variable Selection Methods
  • Current statistical variable selection methods focus on finding good predictors of the response
  • Also need variables to help determine which treatment is best for which types of patients, e.g. tailoring variables
  • Experts typically have knowledge on which variables are good predictors, but intuition about tailoring variables is often lacking
what is a tailoring variable
What is a Tailoring Variable?
  • Tailoring variables help us determine which treatment is best
  • Tailoring variables qualitatively interact with the treatment; different values of the tailoring variable result in different best treatments.

No Interaction Non-qualitative Interaction Qualitative interaction

qualitative interactions
Qualitative Interactions
  • Qualitative interactions have been discussed by many within stat literature (e.g. Byar & Corle,1977; Peto, 1982; Shuster & Van Eys, 1983; Gail & Simon, 1985; Yusuf et al., 1991; Senn, 2001; Lagakos, 2001)
  • Many express skepticism concerning validity of qualitative interactions when found in studies
  • Our approach for finding qualitative interactions should be robust to finding spurious results
qualitative interactions1
Qualitative Interactions
  • We focus on two important factors
    • The magnitude of the interaction between the variable and the treatment indicator
    • The proportionof patients for whom the best choice of treatment changes given knowledge of the variable

big interaction small interaction big interaction

big proportion big proportion small proportion

ranking score s
Ranking Score S
  • Ranking Score:


  • S estimates the quantity described by Parmigiani (2002) as the value of information.
ranking score s1
Ranking Score S
  • Higher Sscorescorrespond to higher evidence of a qualitative interaction between X and A
  • We use this ranking in a variable selection algorithm to select important tailoring variables.
    • Avoid over-fitting in due to large number of X variables
    • Consider variables jointly
variable selection algorithm
Variable Selection Algorithm
  • Select important predictors of R from (X, X*A) using Lasso

-- Select tuning parameter using BIC

  • Select all X*A variables with nonzero S.

-- Use predictors from 1. to form linear regression estimator of to form S.

(using linear models)

  • Lasso on (X, A, XA) (Tibshirani, 1996)
    • Lasso minimization criterion:

where Zi is the vector of predictors for patient i, λ is a penalty parameter

    • Coefficient for A not penalized
    • Value of λ chosen by Bayesian Information Criterion (BIC) (Zou, Hastie & Tibshirani, 2007)
variable selection algorithm1
Variable Selection Algorithm
  • Rank order (X, X*A)variables selected in steps 1 & 2 using a weighted Lasso

-- Weight is 1 if variable is not an interaction

-- Otherwise weight for kth interaction is

-- is a small positive number.

-- Produces a combined ranking of the selected (X, X*A)variables (say p variables).

variable selection algorithm2
Variable Selection Algorithm
  • Choose between variable subsets using a criterion that trades off maximal value of information and complexity.

-- The ordering of the p variables creates p subsets of variables. Estimate the value of information for each of the p subsets

-- Select the subset, k with largest

  • Data simulated under wide variety of realistic decision making scenarios (with and without qualitative interactions)
    • Used X from the CBASP study, generated new Aand R
  • Compared:
    • New method: S with variable selection algorithm
    • Standard method: BIC Lasso on (X, A, XA)
  • 1000 simulated data sets: recorded percentage of time each variable’s interaction with treatment was selected for each method
simulation results
Simulation Results

* Over the total possible increase; 1000 data sets each of size 440

simulation results1
Simulation Results
  • Pros: when the model contained qualitative interactions, the new method gave significant increases in expected response over BIC-Lasso
  • Cons: the new method resulted in a slight increase in the number of spurious interactions over BIC-Lasso
nefazodone cbasp trial
Nefazodone - CBASP Trial

Aim of the Nefazodone CBASP trial – to compare efficacy of three alternate treatments for major depressive disorder (MDD):

  • Nefazodone,
  • Cognitive behavioral-analysis system of psychotherapy (CBASP)
  • Nefazodone + CBASP

Which variables might help tailor the depression treatment to each patient?

nefazodone cbasp trial1
Nefazodone - CBASP Trial
  • For our analysis we used data from 440 patients with
method application and confidence measures
Method Application and Confidence Measures
  • When applying new method to real data it is desirable to have a measure of reliability and to control family-wise error rate
  • We used bootstrap sampling to assess reliability
    • On each of 1000 bootstrap samples:
      • Run variable selection method
      • Record the interaction variables selected
    • Calculate selection percentages over bootstrap samples
error rate thresholds
Error Rate Thresholds
  • To help control family-wise error rate, compute the following inclusion thresholdsfor selection percentages:
    • Repeat 100 times
      • Permute interactions to remove effects from the data
        • Run method on 1000 bootstrap samples of permuted data
        • Calculate selection percentages over bootstrap samples
      • Record largest selection percentage over the p interactions
    • Threshold: (1-α)th percentile over 100 max selection percentages
  • Select all interactions with selection percentage greater than threshold
error rate thresholds1
Error Rate Thresholds
  • When tested in simulations using new method, error rate threshold effectively controlled family-wise error rate
  • This augmentation of bootstrap sampling and thresholding was also tested on BIC Lasso and effectively controlled family-wise error rate in simulations
  • This method provides a list of potential tailoring variables while reducing the number of false leads.
  • Replication is required to confirm the usefulness of a tailoring variable.
  • Our long term goal is to generalize this method so that it can be used with data from Sequential, Multiple Assignment, Randomized Trials as illustrated by STAR*D.
Email Susan Murphy at for more information!
  • This seminar can be found at


  • Support: NIDA P50 DA10075, NIMH R01 MH080015 and NSF DMS 0505432
  • Thanks for technical and data support go to
    • A. John Rush, MD, Betty Jo Hay Chair in Mental Health at the University of Texas Southwestern Medical Center, Dallas
    • Martin Keller and the investigators who conducted the trial `A Comparison of Nefazodone, the Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination for Treatment of Chronic Depression’
lasso weighting scheme
Lasso Weighting Scheme
  • Lasso minimization criterion equivalent to:

so smaller wj means greater importance

  • Weights where
    • vj = 1for predictive variables
    • vj = for prescriptive variables
agv criterion
AGV Criterion
  • For a subset of k variables, X{k} the Average Gain in Value ( AGV) criterion is


  • The criterion selects the subset of variables with the maximum proportion of increase in E[R] per variable
simulation results s score
Simulation Results (S-score)

×Qualitative Interaction

Spurious Interaction

×Qualitative Interaction

Non-qualitative Interaction

Spurious Interaction