1 / 23

ELVIRA II

ELVIRA II. San Sebastián Meeting May, 2004 Andrés Masegosa. Naive-Bayes classifier for gene expression data. Classifier with Continuous Variables Feature Selection Wrapper Search Method. Index. 4. Implementation 4.1 Implemented Classes 4.2 Methods 4.3 Results 5. Conclusions

joshwa
Download Presentation

ELVIRA II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ELVIRA II San Sebastián Meeting May, 2004 Andrés Masegosa San Sebastián Meeting May, 2004

  2. Naive-Bayes classifier for gene expression data Classifier with Continuous Variables Feature Selection Wrapper Search Method San Sebastián Meeting May, 2004

  3. Index 4.Implementation 4.1 Implemented Classes 4.2 Methods 4.3 Results 5.Conclusions 6.Future Works 1.The Naive-Bayes Classifier 1.1 Hipotesis for the creation of the NB classfier. 1.2 Description of a NB classifier. 2.Previous Works 2.1 The lymphoma data set 2.2 Wright et al. paper 3.Selective Gaussian Naive-Bayes 3.1 Anova Phase 3.2 Search Phase 3.3 Stop Condition San Sebastián Meeting May, 2004

  4. The model is simple Advantages It has constraints. It can’t have discrete childs. Drawbacks It hasn’t constraints. It could have discrete childs. It’s possible exact propagation. It has better adjustement possibilities to general distribution. Advantages The model is more complex Drawbacks Hypothesis for the creation of the Naive-Bayes classifier • Hipothesis 1: “The attribute variables are independent given the class variable”. • Hipothesis 2: The attribute variables are distributed as: • Normal Distribution given the class variable: Andrés • Linear Exponential Mixtures given the class variable: Javi San Sebastián Meeting May, 2004

  5. Naive-Bayes Classifier • There are three basic steps in the construction of the Naive–Bayes classifier: • Step 1: Structural Learning  It’s learned the classifier structure. The naive-Bayes model has only arcs from the class variable to the predictives variables, it’s assumed that the predicitives variables are independent given the class. • Step 2: Parametric Learning It’s consists in estimating the distribition for each predictive variable. • Step 3: Propagation Method  It’s carries out the prediction of the class variable given the predictives variables. In our case, the known predictive variables are observed in the bayesian network and after a propagation method (Variable Elmination) is runned to get an á posteriori’ distribution of the class variable. The class with the greatest probability is the predicted value. San Sebastián Meeting May, 2004

  6. Navi-Bayes with MTE • It’s learned a MTE for each predictive variable given the class. • An example: NB classifier learned from Iris data base Estimating a Normal distribution with a MTE San Sebastián Meeting May, 2004

  7. Lymphoma data set IAlizadeth et al (2000) http://llmpp.nih.gov • Hierarchical clustering of gene expression data. Depicted are the ,1.8 million measurements of gene expression from 128 microarray analyses of 96 samples of normal and malignant lymphocytes. • The dendrogram at the left lists the samples studied and provides a measure of the relatedness of gene expression in each sample. The dendrogram is colour coded according to the category of mRNA sample studied (see upper right key). • Each row represents a separate cDNA clone on the microarray and each column a separate mRNA sample. • The results presented represent the ratio of hybridization of fluorescent cDNA probes prepared from each experimental mRNA samples to a reference mRNA sample. These ratios are a measure of relative gene expression in each experimental sample and were depicted according to the colour scale shown at the bottom. • As indicated, the scale extends from ¯uorescence ratios of 0.25 to 4 (-2 to +2 in log base 2 units). Grey indicates missing or excluded data. San Sebastián Meeting May, 2004

  8. Lymphoma data set IIhttp://llmpp.nih.gov • Alizadeh et al (2000): It’s proposed a partition of the diffuse large B-cell lymphoma cases in two clusters by the gene expression profiling: • Germinal Centre B-like: High survival rate. • Activated B-like: Low survival rate. • Rosenwald et al (2002): It’s proposed a new partition of the diffuse large B-cell lymphoma cases in three clusters (274 patients): • Germinal Centre B-like (GCB): High survival rate (134 patients). • Activated B-cell (ABC):Low survival rate ( 83 patients). • Type 3 (TypeIII): Medium survival rate (57 patients). • Wright et al (2003): It’s proposed a Bayesian predictor that estimates the probability of memebership in one of two cancer subgroups (GCB or ABC), with data set of Rosenewald et al. San Sebastián Meeting May, 2004

  9. Wright et al (2003) • Gene Expression Data:http://llmpp.nih.gov/DLBCLpredictor • 8503 genes. • 134 cases of GCB, 83 cases of ABC and 57 cases of Type III. • DLBC subgroup predictor: • Linear Predictor Score: LPS(X)=X = (X1, X2, ...., Xn) • Only K genes with the most significant t statistics were used to form the LPS, the optimal k was determined by a leave one out method. A model including 27 genes had the lowest average error rate. •  where N(x, , ) represents a Normal density function with mean  and desviation . • Training set: 67 GCB + 42 ABC. Validation set: 67 GCB + 41 ABC + 57 Type III. San Sebastián Meeting May, 2004

  10. Wright et al (2003). • This predictor choses a cutoff of 90% certainty. The samples for which there was <90% probability of being in either subgroup are termed ‘unclassified’. • Results: San Sebastián Meeting May, 2004

  11. Index 3.3 Stop Condition 3.4 The M best Explanations 4.Implementation 4.1 Implemented Classes 4.2 Methods 4.3 Results 5.Conclusions 6.Future Works 1.The Naive Bayes Classifier 1.1 Hipotesis for the creation of the NB classfier. 1.2 Description of a NB classifier. 2.Previous Works 2.1 The lymphoma data set 2.2 Wright et al paper 3.Selective Gaussian Naive-Bayes 3.1 Anova Phase 3.2 Search Phase San Sebastián Meeting May, 2004

  12. Selective Gaussian Naive Bayes • It’s a modified wrapper method to construct an optimal Naive Bayes classifier with a minimum number of predictive genes. • The main steps of the algorithm are: • Step 1: First feature selection procedure. Selection of the most not correlated significant variables (Anova phase). • Step 2: Application of a wrapper search method to select the final subset of variables that minimizes the training error rate (Search Phase). • Step 3: Learning of the distribution for each selected gene (Parametric Phase). San Sebastián Meeting May, 2004

  13. Space of the genes Selected gene Cluster of genes Anova Phase: Analisys of Variance. • A dispersion measurement is established for each gene  Anova(X)  [0,+[. • The gene set is preordered from higher to lower Anova measurement. • The gene set is partitioned in K gene subsets where each gene pair of a subset has a correlation coefficient greater than a given bound, U. • For each gene subset, the variable with the greatest anova coefficient is selected. So, we get only k variables of the initial training set. San Sebastián Meeting May, 2004

  14. Search Phase:A wrapper method • Preliminary: • Let A(m,n) the original training set, with m distinct cases and n features for each case. • Leat B(m,k), the projection of A over the k selected genes, the K set in the previous phase. B(m,k) =  (B(m,n),K). • Let KFC(B(m,k)), the error rate obtained with a simple Naive-Bayes classifier evaluation over B by using a T-fold-cross-validation procedure. • Algorithm: • Let P= and Q={X1,...., Xk}. • While (not stopCondition(#(P),r,r) ) • Sea i=indMin{KFC((B(m,k), P  {Xj})): Xj Q} • r=KFC(B(m,k),P) • P=P{Xi}, Q=Q/{Xi}. • r=KFC(B(m,k),P) – r. San Sebastián Meeting May, 2004

  15. 1 r False 1 r True True 0 #(P) False #(P) 0 Avoid overfitting OR Avoid early stopping Search Phase:Stop Condition • The parameters are: #(P), number of elements of P; r, actual error rate; r, increment error rate). • General stop condition:r0 or r>0. • Problems: • Early stopping (only 3-5 genes are selected) with r0. • OverFitting with r>0. • Implemented stop condition: San Sebastián Meeting May, 2004

  16. The M best explanations. Abductive Inference • Due to the high dimensionality of the gene expression data sets, it’s usually use cross validation methods to estimate the train error rate of a classification model. • If a T-fold-cross method is used, T final gene subsets are got by the wrapper search method. The question is: how do I select a unique gene subset to apply to the validation data set?. • Method: • Let Ci , i  {1, ..., T}, the subset returned by the wrapper method in the phase i of the cross validation procedure. • Let C=  Ci y N = #(C) • Let Z a data base of T cases, where the case ‘j’ is a tuple : {a1, ..., aN}, with ai = 1, if Xi Ci; ai=0,otherwise. • Let B a BN learned by a K2 learning method. An abductive inference method returns the M most probablity explanations of the BN that is equivalent to get the M most probable gene subsets. • The final subset is the subset with minimum leaving one out training error rate. San Sebastián Meeting May, 2004

  17. Implementation I • Included in the ‘learning.classification.supervised.mixed’ package. • This package containsthe follow classes: • MixedClassifer class It’s a abstract class. It was designed to be the parent of all the mixed classifiers to implement. It inherits from DiscreteClassifier class. • Mixed_Naive_Bayes class  It’s a public class to learn a mixed Naive-Bayes classfication model. It inherits form MixedClassifer class. This class implements the structural learning and the selective structural learning methods. This last method contains the implementation of our wrapper search method and it needs to define the following methods (to be implemented later): • double evaluationKFC(DataBaseCases cases, int numclass). • double evaluationLOO (DataBaseCases cases, int numclass). • Bnet getNewClassifier(DataBaseCases cases). San Sebastián Meeting May, 2004

  18. Implementation II • Gaussian_Naive_Bayes class  It’s a public class that implements the parametric learning of a mixed NB classifer. It’s assumed that the predictive variables are ditributed as a normal distribution given the class. It inherits from Mixed_Naive_Bayes class. • Selective_GNB class  It’s a public class that implements a gaussian naive bayes with feature selection. So, this class: • Implements the Selective_Classifier interface. • Overwrites the following methods: • structuralLearnig, now this method call to selectiveStructuralLearning metod. • And evaluationKFC, evaluationLOO, getNewClassifier methods. • Selective_Classifier interface It’s a public interface for define variable selection in a classifier method. San Sebastián Meeting May, 2004

  19. Methods • 10 training and validation sets were randomly generated. • The three phases were applied to each training set. • The parameters were: • 10 fold cross validation. • M was fixed to 20. • U was fixed to 0.15. • This predictor chose a cutoff of 80% certainty. The samples for which there was <80% likelihood of being in either subgroup were termed ‘unclassified’. • The stop condition was implemented as: • Avoid overfitting: r > n*u2/n2 ; u2=0.03 and n2=20. • Avoid earlier stopping: incRate < (n1-n)*u1/n1 ; u1=0.1, n1=10. San Sebastián Meeting May, 2004

  20. Model Prediction DLBCL subgroup Training set Validation set Results I • Phase Anova: (Confidence Intervals at 95%) • Size (gene number): [74.3, 83.1] • Train accuracy rate (%): [96.8, 98.6] • Test accuracy rate (%): [92.8, 95.4] • Test -log likelihood: [41.6, 72.3] • TypeIII Test accuracy rate (%): [17.75, 18.66] • TypeIII Test -log likelihood: ‘Infinity’ San Sebastián Meeting May, 2004

  21. Model Prediction DLBCL subgroup Training set Validation set Results II • Phase Anova+ Phase Search: (Confidence Intervals at 95%) • Size (gene number): [6.17, 7.82] • Train accuracy rate (%): [95.2, 98.0] • Test accuracy rate (%): [88.83, 91.9] • Test -log likelihood: [26.72, 40.46] • TypeIII Test accuracy rate (%): [20.0, 26.2] • TypeIII Test -log likelihood: [214.0, 264.6] San Sebastián Meeting May, 2004

  22. Conclusions • It’s a simple classification method that provides good results. • Its main problem is that, due to the search process, the train error rate goes down quickly and the mean number of selected genes is too low (around eight genes). • Altough this trend is corrected by the anova phase, the k-fold-cross validation and the flexible stop condition. • Get the M best explanations is a very good technique to fuse several groups of extracted genes by a feature selection method. San Sebastián Meeting May, 2004

  23. Future works • Develop more sophisticated models: • Include replacement variables to manage losen data. • Consider Multidimensionals Gaussian distributions. • Improve the MTE Gaussian Naive Bayes model. • Apply this model to other data sets as breast cancer, colon cancer ... • Compare with other models with discrete variables. San Sebastián Meeting May, 2004

More Related