Development and validation of prognostic classifiers using high dimensional data
This presentation is the property of its rightful owner.
Sponsored Links
1 / 127

Development and Validation of Prognostic Classifiers using High Dimensional Data PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on
  • Presentation posted in: General

Development and Validation of Prognostic Classifiers using High Dimensional Data. Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov. Class Prediction. Predict which tumors will respond to a particular treatment

Download Presentation

Development and Validation of Prognostic Classifiers using High Dimensional Data

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Development and validation of prognostic classifiers using high dimensional data

Development and Validation of Prognostic Classifiers using High Dimensional Data

Richard Simon, D.Sc.

Chief, Biometric Research Branch

National Cancer Institute

http://brb.nci.nih.gov


Class prediction

Class Prediction

  • Predict which tumors will respond to a particular treatment

  • Predict which patients will relapse after a particular treatment


Class prediction1

Class Prediction

  • A set of genes is not a classifier

  • Testing whether analysis of independent data results in selection of the same set of genes is not an appropriate test of predictive accuracy of a classifier


Components of class prediction

Components of Class Prediction

  • Feature (gene) selection

    • Which genes will be included in the model

  • Select model type

    • E.g. Diagonal linear discriminant analysis, Nearest-Neighbor, …

  • Fitting parameters (regression coefficients) for model

    • Selecting value of tuning parameters

  • Estimating prediction accuracy


Class prediction class comparison

Class Prediction ≠ Class Comparison

  • The criteria for gene selection for class prediction and for class comparison are different

    • For class comparison false discovery rate is important

    • For class prediction, predictive accuracy is important

  • Demonstrating statistical significance of prognostic factors is not the same as demonstrating predictive accuracy.

  • Statisticians are used to inference, not prediction

  • Most statistical methods were not developed for p>>n prediction problems


Feature gene selection

Feature (Gene) Selection

  • Select genes that are differentially expressed among the classes at a significance level  (e.g. 0.01)

    • The  level is a tuning parameter which can be optimized by “inner” cross-validation

    • For class comparison false discovery rate is important

    • For class prediction, predictive accuracy is important

    • For prediction it is usually more serious to exclude an informative variable than to include some noise variables

  • L1 penalized likelihood or predictive likelihood for gene selection with penalty value selected by cross-validation

    • Select smallest number of genes consistent with accurate prediction


Development and validation of prognostic classifiers using high dimensional data

Optimal significance level cutoffs for gene selection. 50 differentially expressed genes out of 22,000 on n arrays


Complex gene selection

Complex Gene Selection

  • Small subset of genes which together give most accurate predictions

    • Genetic algorithms

  • Little evidence that complex feature selection is useful in microarray problems

    • Failure to compare to simpler methods

    • Improper use of cross-validation


Linear classifiers for two classes

Linear Classifiers for Two Classes


Linear classifiers for two classes1

Linear Classifiers for Two Classes

  • Fisher linear discriminant analysis

  • Diagonal linear discriminant analysis (DLDA) assumes features are uncorrelated

  • Compound covariate predictor (Radmacher)

  • Golub’s weighted voting method

  • Support vector machines with inner product kernel


Fisher lda

Fisher LDA


The compound covariate predictor ccp

The Compound Covariate Predictor (CCP)

  • Motivated by J. Tukey, Controlled Clinical Trials, 1993

  • A compound covariate is built from the basic covariates (log-ratios)

    tj is the two-sample t-statistic for gene j.

    xijis the log-expression measure of sample i for gene j.

    Sum is over selected genes.

  • Threshold of classification: midpoint of the CCP means for the two classes.


Linear classifiers for two classes2

Linear Classifiers for Two Classes

  • Compound covariate predictor

    Instead of for DLDA


Support vector machine

Support Vector Machine


Other simple methods

Other Simple Methods

  • Nearest neighbor classification

  • Nearest k-neighbors

  • Nearest centroid classification

  • Shrunken centroid classification


Nearest neighbor classifier

Nearest Neighbor Classifier

  • To classify a sample in the validation set as being in outcome class 1 or outcome class 2, determine which sample in the training set it’s gene expression profile is most similar to.

    • Similarity measure used is based on genes selected as being univariately differentially expressed between the classes

    • Correlation similarity or Euclidean distance generally used

  • Classify the sample as being in the same class as it’s nearest neighbor in the training set


When p n

When p>>n

  • It is always possible to find a set of features and a weight vector for which the classification error on the training set is zero.

  • Why consider more complex models?


Development and validation of prognostic classifiers using high dimensional data

  • Some predictive classification problems are easy

    • Toy problems

    • Most all methods work well

    • It is possible to predict accurately with few variables

  • Many real classification problems are difficult

    • Difficult to distinguish informative variables from the extreme order statistics of noise variables

    • Comparative studies generally indicate that simpler methods work as well or better for microarray problems because they avoid overfitting the data.


Other methods

Other Methods

  • Top-scoring pairs

  • CART

  • Random Forrest


Dimension reduction methods

Dimension Reduction Methods

  • Principal component regression

  • Supervised principal component regression

  • Partial least squares

  • Compound covariate and DLDA

  • Supervised clustering

  • L1 penalized logistic regression


When there are more than 2 classes

When There Are More Than 2 Classes

  • Nearest neighbor type methods

  • Decision tree of binary classifiers


Decision tree of binary classifiers

Decision Tree of Binary Classifiers

  • Partition the set of classes {1,2,…,K} into two disjoint subsets S1 and S2

    • e.g. S1={1}, S2 ={2,3,4}

    • Develop a binary classifier for distinguishing the composite classes S1 and S2

      • Compute the cross-validated classification error for distinguishing S1 and S2

  • Repeat the above steps for all possible partitions in order to find the partition S1and S2 for which the cross-validated classification error is minimized

  • If S1and S2 are not singleton sets, then repeat all of the above steps separately for the classes in S1and S2 to optimally partition each of them


Development and validation of prognostic classifiers using high dimensional data

cDNA Microarrays

Parallel Gene Expression Analysis

Gene-Expression Profiles in Hereditary Breast Cancer

  • Breast tumors studied:

    • 7 BRCA1+ tumors

    • 8 BRCA2+ tumors

    • 7 sporadic tumors

  • Log-ratios measurements of 3226 genes for each tumor after initial data filtering

RESEARCH QUESTION

Can we distinguish BRCA1+ from BRCA1– cancers and BRCA2+ from BRCA2– cancers based solely on their gene expression profiles?


Brca1

BRCA1


Brca2

BRCA2


Classification of brca2 germline mutations

Classification of BRCA2 Germline Mutations


Evaluating a classifier

Evaluating a Classifier

  • “Prediction is difficult, especially the future.”

    • Neils Bohr


Evaluating a classifier1

Evaluating a Classifier

  • Fit of a model to the same data used to develop it is no evidence of prediction accuracy for independent data

    • Goodness of fit vs prediction accuracy


Development and validation of prognostic classifiers using high dimensional data

  • Hazard ratios and statistical significance levels are not appropriate measures of prediction accuracy

  • A hazard ratio is a measure of association

    • Large values of HR may correspond to small improvement in prediction accuracy

  • Kaplan-Meier curves for predicted risk groups within strata defined by standard prognostic variables provide more information about improvement in prediction accuracy

  • Time dependent ROC curves within strata defined by standard prognostic factors can also be useful


Time dependent roc curve

Time-Dependent ROC Curve

  • Sensitivity vs 1-Specificity

  • Sensitivity = prob{M=1|S>T}

  • Specificity = prob{M=0|S<T}


Validation of a predictor

Validation of a Predictor

  • Internal validation

    • Re-substitution estimate

      • Very biased

    • Split-sample validation

    • Cross-validation

  • Independent data validation


Validating a predictive classifier

Validating a Predictive Classifier

  • Fit of a model to the same data used to develop it is no evidence of prediction accuracy for independent data

    • Goodness of fit is not prediction accuracy

  • Demonstrating statistical significance of prognostic factors is not the same as demonstrating predictive accuracy

  • Demonstrating stability of selected genes is not demonstrating predictive accuracy of a model for independent data


Split sample evaluation

Split-Sample Evaluation

  • Training-set

    • Used to select features, select model type, determine parameters and cut-off thresholds

  • Test-set

    • Withheld until a single model is fully specified using the training-set.

    • Fully specified model is applied to the expression profiles in the test-set to predict class labels.

    • Number of errors is counted

    • Ideally test set data is from different centers than the training data and assayed at a different time


Development and validation of prognostic classifiers using high dimensional data

log-expression ratios

training set

specimens

test set

Cross-Validated Prediction (Leave-One-Out Method)

1. Full data set is divided into training and test sets (test set contains 1 specimen).

2. Prediction rule is built from scratch using the training set.

3. Rule is applied to the specimen in the test set for class prediction.

4. Process is repeated until each specimen has appeared once in the test set.


Leave one out cross validation

Leave-one-out Cross Validation

  • Omit sample 1

    • Develop multivariate classifier from scratch on training set with sample 1 omitted

    • Predict class for sample 1 and record whether prediction is correct


Leave one out cross validation1

Leave-one-out Cross Validation

  • Repeat analysis for training sets with each single sample omitted one at a time

  • e = number of misclassifications determined by cross-validation

  • Subdivide e for estimation of sensitivity and specificity


Development and validation of prognostic classifiers using high dimensional data

  • Cross validation is only valid if the test set is not used in any way in the development of the model. Using the complete set of samples to select genes violates this assumption and invalidates cross-validation.

  • With proper cross-validation, the model must be developed from scratch for each leave-one-out training set. This means that feature selection must be repeated for each leave-one-out training set.

    • Simon R, Radmacher MD, Dobbin K, McShane LM. Pitfalls in the analysis of DNA microarray data. Journal of the National Cancer Institute 95:14-18, 2003.

  • The cross-validated estimate of misclassification error is an estimate of the prediction error for model fit using specified algorithm to full dataset


Development and validation of prognostic classifiers using high dimensional data

Permutation Distribution of Cross-validated Misclassification Rate of a Multivariate ClassifierRadmacher, McShane & SimonJ Comp Biol 9:505, 2002

  • Randomly permute class labels and repeat the entire cross-validation

  • Re-do for all (or 1000) random permutations of class labels

  • Permutation p value is fraction of random permutations that gave as few misclassifications as e in the real data


Prediction on simulated null data

Prediction on Simulated Null Data

  • Generation of Gene Expression Profiles

  • 14 specimens (Pi is the expression profile for specimen i)

  • Log-ratio measurements on 6000 genes

  • Pi ~ MVN(0, I6000)

  • Can we distinguish between the first 7 specimens (Class 1) and the last 7 (Class 2)?

  • Prediction Method

  • Compound covariate prediction (discussed later)

  • Compound covariate built from the log-ratios of the 10 most differentially expressed genes.


Major flaws found in 40 studies published in 2004

Major Flaws Found in 40 Studies Published in 2004

  • Inadequate control of multiple comparisons in gene finding

    • 9/23 studies had unclear or inadequate methods to deal with false positives

      • 10,000 genes x .05 significance level = 500 false positives

  • Misleading report of prediction accuracy

    • 12/28 reports based on incomplete cross-validation

  • Misleading use of cluster analysis

    • 13/28 studies invalidly claimed that expression clusters based on differentially expressed genes could help distinguish clinical outcomes

  • 50% of studies contained one or more major flaws


Class prediction2

Class Prediction

  • Cluster analysis is frequently used in publications for class prediction in a misleading way


Fallacy of clustering classes based on selected genes

Fallacy of Clustering Classes Based on Selected Genes

  • Even for arrays randomly distributed between classes, genes will be found that are “significantly” differentially expressed

  • With 10,000 genes measured, about 500 false positives will be differentially expressed with p < 0.05

  • Arrays in the two classes will necessarily cluster separately when using a distance measure based on genes selected to distinguish the classes


Development and validation of prognostic classifiers using high dimensional data

Myth

  • Split sample validation is superior to LOOCV or 10-fold CV for estimating prediction error


Comparison of internal validation methods molinaro pfiffer simon

Comparison of Internal Validation MethodsMolinaro, Pfiffer & Simon

  • For small sample sizes, LOOCV is much less biased than split-sample validation

  • For small sample sizes, LOOCV is preferable to 10-fold or 5-fold cross-validation or repeated k-fold versions

  • For moderate sample sizes, 10-fold is preferable to LOOCV


Simulated data 40 cases 10 genes selected from 5000

Simulated Data40 cases, 10 genes selected from 5000


Simulated data 40 cases

Simulated Data40 cases


Sample size planning references

Sample Size Planning References

  • K Dobbin, R Simon. Sample size determination in microarray experiments for class comparison and prognostic classification. Biostatistics 6:27, 2005

  • K Dobbin, R Simon. Sample size planning for developing classifiers using high dimensional DNA microarray data. Biostatistics 8:101, 2007

  • K Dobbin, Y Zhao, R Simon. How large a training set is needed to develop a classifier for microarray data? Clinical Cancer Res 14:108, 2008


Development of empirical gene expression based classifier

Development of Empirical Gene Expression Based Classifier

  • 20-30 phase II responders are needed to compare to non-responders in order to develop signature for predicting response


Sample size planning for classifier development

Sample Size Planning for Classifier Development

  • The expected value (over training sets) of the probability of correct classification PCC(n) should be within  of the maximum achievable PCC()


Probability model

Probability Model

  • Two classes

  • Log expression or log ratio MVN in each class with common covariance matrix

  • m differentially expressed genes

  • p-m noise genes

  • Expression of differentially expressed genes are independent of expression for noise genes

  • All differentially expressed genes have same inter-class mean difference 2

  • Common variance for differentially expressed genes and for noise genes


Classifier

Classifier

  • Feature selection based on univariate t-tests for differential expression at significance level 

  • Simple linear classifier with equal weights (except for sign) for all selected genes. Power for selecting each of the informative genes that are differentially expressed by mean difference 2 is 1-(n)


Development and validation of prognostic classifiers using high dimensional data

  • For 2 classes of equal prevalence, let 1 denote the largest eigenvalue of the covariance matrix of informative genes. Then


Development and validation of prognostic classifiers using high dimensional data

Sample size as a function of effect size (log-base 2 fold-changebetween classes divided by standard deviation). Two different tolerances shown, . Each class is equally represented in the population. 22000 genes on an array.


Developing and validating classifiers of survival risk

Developing and Validating Classifiers of Survival Risk


Brb arraytools survival risk group prediction

BRB-ArrayToolsSurvival Risk Group Prediction

  • No need to transform data to good vs bad outcome. Censored survival is directly analyzed

  • Gene selection based on significance in univariate Cox Proportional Hazards regression

  • Uses k principal components of selected genes

  • Gene selection re-done for each resampled training set

  • Develop k-variable Cox PH model for each leave-one-out training set


Brb arraytools survival risk group prediction1

BRB-ArrayToolsSurvival Risk Group Prediction

  • Classify left out sample as above or below median risk based on model not involving that sample

  • Repeat, leaving out 1 sample at a time to obtain cross-validated risk group predictions for all cases

  • Compute Kaplan-Meier survival curves of the two predicted risk groups

  • Permutation analysis to evaluate statistical significance of separation of K-M curves


Brb arraytools survival risk group prediction2

BRB-ArrayToolsSurvival Risk Group Prediction

  • Compare Kaplan-Meier curves for gene expression based classifier to that for standard clinical classifier

  • Develop classifier using standard clinical staging plus genes that add to standard staging


Does an expression profile classifier predict more accurately than standard prognostic variables

Does an Expression Profile Classifier Predict More Accurately Than Standard Prognostic Variables?

  • Not an issue of which variables are significant after adjusting for which others or which are independent predictors

    • Predictive accuracy and inference are different

  • The predictiveness of the expression profile classifier can be evaluated within levels of the classifier based on standard prognostic variables


Survival risk group prediction

Survival Risk Group Prediction

  • LOOCV loop:

    • Create training set by omitting i’th case

  • Develop PH model for training set

  • Compute predictive index for i’th case using PH model developed for training set

  • Compute percentile of predictive index for i’th case among predictive indices for cases in the training set


Survival risk group prediction1

Survival Risk Group Prediction

  • Plot Kaplan Meier survival curves for cases with predictive index percentiles above 50% and for cases with cross-validated risk percentiles below 50%

    • Or for however many risk groups and thresholds is desired

  • Compute log-rank statistic comparing the cross-validated Kaplan Meier curves


Survival risk group prediction2

Survival Risk Group Prediction

  • Evaluate individual genes by fitting single variable proportional hazards regression models to log expression for each gene

  • Select genes based on p-value threshold for single gene PH regressions

  • Compute first k principal components of the selected genes

  • Fit PH regression model with the k pc’s as predictors. Let b1 , …, bk denote the estimated regression coefficients

  • To predict for case with expression profile vector x, compute the k supervised pc’s y1 , …, yk and the predictive index  = b1 y1 + … + bk yk


Survival risk group prediction3

Survival Risk Group Prediction

  • Repeat the entire procedure for permutations of survival times and censoring indicators to generate the null distribution of the log-rank statistic

    • The usual chi-square null distribution is not valid because the cross-validated risk percentiles are correlated among cases

  • Evaluate statistical significance of the association of survival and expression profiles by referring the log-rank statistic for the unpermuted data to the permutation null distribution


Development and validation of prognostic classifiers using high dimensional data

R. Simon, G. Bianchini, M. Zambetti, S. Govi, G. Mariani, M. L. Carcangiu, P. Valagussa, L. Gianni

National Cancer Institute, Bethesda, MD; Fondazione IRCCS - Istituto Tumori di Milano, Milan, Italy

Outcome prediction in estrogen-receptor positive, chemotherapy and tamoxifen treated patients with locally advanced breast cancer


Patients and methods i

PATIENTS AND METHODS - I

  • Fifty-seven patients with ER positive tumors enrolled in a neoadjuvant clinical trial for LABC were evaluated. All patients had been treated with doxorubicin and paclitaxel q 3wk x 3, followed by weekly paclitaxel x 12 before surgery, then adjuvant intravenous CMF q 4wk x 4 and thereafter tamoxifen.

  • High-throughput qRT-PCR gene expression analysis in paraffin-embedded formalin-fixed core biopsies at diagnosis was performed by Genomic Health to quantify expression of 363 genes (plus 21 for Oncotype DXTM determination), as described previously (Gianni L, JCO 2005). RS genes were excluded from analysis.


Patients and methods ii

PATIENTS AND METHODS - II

  • Three models (prognostic index) were developed to predict Distant Event Free Survival (DEFS):

    • GENE MODEL Using only expression data, genes were selected based on univariate Cox analysis p value under a specific threshold significance level.

    • COVARIATES MODEL Using RS (as continuous variable), age and IBC status (covariates) a multivariate proportional hazards model was developed.

    • COMBINED MODEL Using a combination of these covariates and expression data, genes were selected which add to predicting survival over the predictive value provided by the covariates and under a specific threshold significance level.

  • Survival risk groups were constructed using the supervised principal component method implemented in BRB-ArrayTools (Bair E, Tibshirani R, PLOS Biology 2004).


Patients and methods iii

PATIENTS AND METHODS - III

  • In order to evaluate the predictive value for each model a complete Leave-One-Out Cross-Validation was used.

    • For each i-th cross-validated training set (with one case removed) a prognostic index (PI) function was created. The PI for the omitted patient is ranked relative to the PI for the i-th training set. Because the PI is a continuous variable, a cut-off percentiles have to be pre-specified for defining the risk groups. The omitted patient is placed into a risk group based on her percentile ranking. The entire procedure has been repeated using different cut-off percentiles (BRB-ArrayTools User’s Manual v3.7).


Patients and methods iv

PATIENTS AND METHODS - IV

  • Statistical significance was determined by repeating the entire cross-validation process 1000 random permutations of the survival data.

    • For GENE MODEL the p value was testing the null hypothesis that there is no relation between the expression data and survival (by providing a null-distribution of the log-rank statistic)

    • For COVARIATES MODEL the p value was the parametric log-rank test statistic between risk groups

    • For COMBINED MODEL the p value addressed whether the expression data adds significantly to risk prediction compared to the covariates


Results patients characteristics at diagnosis

RESULTSPatients characteristics at diagnosis

  • The median follow-up was 76 months (range 18-103) (by inverse Kaplan-Meier method)

  • Patients characteristics were summarized in Table 1.


Os and defs of all patients

OS

DEFS

OS and DEFS of all patients

Overall Survival and Distant Event Free survival – All patients


Genes selected for the gene model and combined model

Genes selected for the GENE MODEL and COMBINED MODEL

  • The significance level for gene selection used for the identified models was p=0.005.

  • All genes included in the COMBINED MODEL were also selected in the GENE MODEL.


Cross validated kaplan meier curves for risk groups using 50th percentile cut off

Cross-validated Kaplan-Meier curves for risk groups using 50th percentile cut-off

DISTANT EVENT FREE SURVIVAL

DISTANT EVENT FREE SURVIVAL

DISTANT EVENT FREE SURVIVAL

COMBINED

MODEL

COVARIATES

MODEL

GENE

MODEL


Development and validation of prognostic classifiers using high dimensional data

An Evaluation of Resampling Methods for Assessment of Survival Risk Prediction in High-dimensional SettingsJyothi Subramanian and Richard Simon*Biometric Research Branch National Cancer Institute,


Development and validation of prognostic classifiers using high dimensional data

ABSTRACT

Resampling techniques are often used to provide an initial assessment of accuracy for prognostic prediction models developed using high-dimensional genomic data with binary outcomes. Risk prediction is most important, however, in medical applications and frequently the outcome measure is a right-censored time-to-event variable such as survival. Although several methods have been developed for survival risk prediction with high-dimensional genomic data, there has been little evaluation of the use of resampling techniques for the assessment of such models. Using real and simulated datasets, we compared several resampling techniques for their ability to estimate the accuracy of risk prediction models. Our study showed that accuracy estimates for popular resampling methods such as sample splitting and leave-one-out cross-validation have a higher mean square error than for other methods. Moreover, the large variability of the split-sample and leave-one-out cross-validation may make the point estimates of accuracy obtained using these methods unreliable and hence should be interpreted carefully. A k-fold cross-validation with k = 5 or 10 was seen to provide a good balance between bias and variability for a wide range of data settings and should be more widely adopted in practice.


Cross validated time dependent roc curve

Cross-Validated Time Dependent ROC Curve


Development and validation of prognostic classifiers using high dimensional data

n = 40

n = 80

n = 160

n = 160

n = 80

n = 40

Figure 1a. Distribution of the estimated resampling and true AUC(t) for SuperPCR. (Top) High-signal data. (Bottom) Null data.


Development and validation of prognostic classifiers using high dimensional data

n = 40

n = 80

n = 160

n = 40

n = 160

n = 80

Figure 1b. Distribution of the estimated resampling and true AUC(t) for SuperPCR. (Top) Lung cancer data. (Bottom) DLBCLdata.


Development and validation of prognostic classifiers using high dimensional data

Figure 2. In the case of the null dataset, the AUC(t) is expected to be around 0.5. The number of times ÂRS(Sn) is too pessimistic (below 0.3, black bars) or too optimistic (above 0.7, light grey bars) in a total of 100 replications for the null dataset is shown. In comparison to this, the number of times Â(Sn) is below 0.3 or above 0.7 is nil. (a) n = 40 (b) n = 80 (c) n = 160.


Development and validation of prognostic classifiers using high dimensional data

Figure 3: The number of times ÂRS(Sn) is too pessimistic (below 0.3, black bars) or too optimistic (above 0.7, light grey bars) in a total of 100 replications for the lung cancer dataset is compared with the number of times Â(Sn) is below 0.3 or above 0.7. (a) n = 40 (b) n = 80 (c)n = 160


Figure 4 bias in the resampling auc t estimates

Figure 4: Bias in the resampling AUC(t) estimates.


Figure 5 mean square error in the resampling auc t estimates

Figure 5: Mean square error in the resampling AUC(t) estimates.


Brb arraytools

Contains analysis tools that I have selected as valid and useful

Analysis wizard and multiple help screens for biomedical scientists

Imports data from all platforms and major databases

Automated import of data from NCBI Gene Express Omnibus

BRB-ArrayTools


Predictive classifiers in brb arraytools

Classifiers

Diagonal linear discriminant

Compound covariate

Bayesian compound covariate

Support vector machine with inner product kernel

K-nearest neighbor

L1 penalized logistic regression

Nearest centroid

Shrunken centroid (PAM)

Random forrest

Tree of binary classifiers for k-classes

Survival risk-group

Supervised pc’s

Feature selection options

Univariate t/F statistic

Hierarchical variance option

Restricted by fold effect

Univariate classification power

Recursive feature elimination

Top-scoring pairs

Validation methods

Split-sample

LOOCV

Repeated k-fold CV

.632+ bootstrap

Predictive Classifiers in BRB-ArrayTools


Selected features of brb arraytools

Selected Features of BRB-ArrayTools

  • Multivariate permutation tests for class comparison to control number and proportion of false discoveries with specified confidence level

    • Permits blocking by another variable, pairing of data, averaging of technical replicates

  • SAM

    • Fortran implementation 7X faster than R versions

  • Extensive annotation for identified genes

    • Internal annotation of NetAffx, Source, Gene Ontology, Pathway information

    • Links to annotations in genomic databases

  • Find genes correlated with quantitative factor while controlling number of proportion of false discoveries

  • Find genes correlated with censored survival while controlling number or proportion of false discoveries

    • Kaplan Meier curves

  • Analysis of variance


Thank you

Thank You

  • I’ve learned things from your questions. I hope that you have too.

  • “When you’re green you’re growing, when your ripe you’re rotting.”

  • Keep learning new things and expanding your boundaries.

  • There are major opportunities for improving public health. Think big and be brave.


  • Login